I would definitely agree that it sounds like something weird is happening. It is still possible that your mother does genuinely have a chronic medical condition, but it sounds like you are at the point where you need to establish concretely that this is the case and get a real answer to the question of whether she is being poisoned. I would find some way to get your mother or the substance you have found tested.
My thinking is that if you report your father to adult protective services without already having evidence in hand that he is poisoning her, he will simply deny it and point to your mother's "auto-immune disorder." If there are no concrete signs of poisoning, the services may not do much (though it may still be worth it to try).
I tried to look up somewhere you could send samples, but other than the link another commenter left below, it sounds like your mother will need to go see a doctor and request blood tests. You may try going with her to see a doctor and explaining your suspicions, and bring a sample of the silver substance. However, this would require your mother going along with it. Otherwise, it sounds like you could go talk to the police and give them what samples you have collected (I'd go with some of the silver stuff, your mother's hair, maybe from a brush, and if you can, a sample of the food your father makes for your mother). If they agree that it sounds strange, they may send what you give them to their forensic labs.
Measuring the weight of the silver substance was also a really good idea, and keeping track of its weight over time to see if it's used is good too. Density is mass divided by volume, generally grams over cubic centimeters. Generally speaking, one fluid ounce is 29.57 cubic centimeters, and 1 mL is 1 cm3, so if the bottle is labeled you could calculate it easily. You could also measure the bottle in centimeters -- the volume of a cylinder is pi * r2 * h. Mercury has a density of 13.534 g/cm3. Based on the measurements you mentioned below, that sounds really close to what you have. But if you really want to know if it's mercury, you will have to have a lab test it.
You could also call an elder abuse hotline like another commenter suggested and ask how they would recommend you approach it. This link has a number of local hotlines: http://www.nccafv.org/state_elder_abuse_hotlines.htm
Overall, though, I would recommend that you do all of this in secret (to the extent that you can). If your father finds out you are doing this, he may destroy evidence or switch to a harder-to-detect method of poisoning.
Sorry this comment got really long, but what you have described is definitely weird, and I think if you have the feeling it's something your father would do, you are probably right. Your instincts are based on real things your father has done, like the "revenge" on his co-worker, and poisoning someone does not sound out of line for a narcissist if they thought it would give them what they wanted (control) and if they thought they were not going to get caught.