I can imagine it must have taken lots of painful hardship. You had to try this and that, and they probably tried it first on animals. And imagine the pain of seeing your pet animal suffer death through poisoning. Well, of course, there were a lot more triumphs, seeing your sick pet get well through your experimented herbals and seeing the same effect on humans. There must have been celebrations and toasts to that effect.
The real wonder is how ancient men first came to the conclusion that plants and herbs could help in healing? Why not try stones or rocks or the soil or urine? Why herbs and plants? Probably, they must have tried these stuff too, and they didn't work--even ended up in disaster (although I heard of one herbalist in Deep Asia who makes her patients drink their urine to cure ailments, even cancer. The idea, she said, was that urine has super healing enzymes). They tried everything and narrowed down their search to plants.
Now, they probably connected the good feeling they had each time they ate herbs and plants and fruits and crops with the possibility that they could heal. It was the first experience of wellness, I guess. They felt well and stronger eating plant produce so they must have concluded that it could also heal and tried them. But how did they end up making herbal medicines with them, and not simply become content at eating herbs and plants? That's part of how did man start discovering safe herbs.
They must have realized that their intake of plants and herbs was limited--they could only eat so much--and getting healing benefits from them meant eating much more than enough of them, especially during sickness. And sickness time is when you tend to eat little, if any. So they thought of juicing or squeezing out extracts by pounding them, boiling, or steaming. That way, you won't have to eat a lot but get more nutrients by merely drinking the extract from loads and loads of the plants. That's probably also when they tried topical uses of their newly discovered herbal medicine.
Up to this day, that's how we do herbal medicine--somewhat the same with the answer to how did man start discovering safe herbs. We continue to try and experiment with animals and then try it with humans. But the availability of the microscope gives nutritionists, pharmacists, and chemists a full advantage over their ancient counterparts. They can identify ahead what is and is not safe before they try it on anyone.