Joking aside, this is going to be difficult without joking, I would say rubber for worms, and plastic for hard baits, though I usually just say the name of the bait (worm, creature, crank, jerk, etc).
Everyone pretty much knows that worms are soft and cranks are hard.
Not gonna joke. Not...