At that age, the only reason to give milk is usually to ensure enough calcium in the diet. If your son is getting enough calcium from other sources (cheese, yogurt, etc), he'll be fine without milk. Don't bother with a toddler formula - it's just marketing making you think it's important. If you choose to give a milk, stick with cow milk or almond milk. Soy milk is very processed and isn't the best for boys due to the estrogen-like effects.
Also, limiting nursing at that age is totally normal, it doesn't mean he'll wean completely (unless that is what you want and you encourage it). He may just drop down to only nursing one or two times a day, which can continue as long as both of you are fine with it. Usually, if he is dropping nursing sessions it is because he is getting nutrients from the solid foods. Keep feeding him and giving him water to drink (or whatever milk you choose) and he'll naturally regulate his nursing sessions. If you do not want to wean, do limit the amount of milk (cow or other) you give him so it doesn't replace the caloric value of your breastmilk.
Bonus tip: avoid giving him drinks in bottles if you can, because some kids will develop an attachment to bottles and you'll then be stuck with bottle weaning at some point.