All the cables and patch cords I know about have either PVC composite or Teflon outer jacket that are essentially plastics, not latex or rubber. However, the PVC composite materials are never disclosed to public, and there is almost no way to tell whether they have some material in particular or not. Most of its weight is PVC (Polyvinyl Chloride), plus some mineral additions that make it burn slower, plus some elastic material, which makes it bend easier. It could not be rubber though.
Another thing that may be of some concern regarding the allergy is dust that collects on cables and cords. If you have no problem with dust, it should not be a problem at all.
Dmitri Abaimov, RCDD