If the picture is high resolution (many pixels wide and high) you can get a long way by just scaling it down to a small enough size that still looks OK. Any simple Graphics editor - even MS Paint using the Image Stretch menu (just set X and Y to the same percentage, less than 100%, and it will shrink the picture) Smaller dimensions will certainly make a smaller file.
A decent Graphics editor (I use
GIMP which is powerful and free) should also have a way of setting the amount of compression while saving an JPEG file. The GIMP will let you slide the compression level up and down on a slider bar and shows the actual file size changing as you do it, also previewing the picture so you can see how much distortion is created by the compression. I think Paint Shop Pro can do something like that too.