What if the primary barrier to species crosses is really a language difference?
It's a given that plants communicate chemically. If they also use a chemical "language" to recognize others of their species for purposes of mating, it could explain a lot about compatibility. If stress alters or weakens part of the unique chemical language of a plant just prior to pollination, it may be that enough similarities exist to override the resistance to producing seed with a distant relative regarldess of chromosome count.
I'm taking a giant leap and assuming that there may be a basic chemical code inherited from an ancient relative still present in the language of all species within a family. If species origin is primarily by hybridization, each species may carry part of the language code from each of the parent species. Mutations and natural variations, over time, may work to alter such codes to the extent that groups of the same species may be imagined to speak the same language but with a different dialect.
It is also possible that some individual plants may have a damaged copy of such a language code especially early generation hybrids where a new species language hasn't had time to become stable. If so, an entire group of plants may use an abbreviated part of the language code making them more likely to cross with additional related species.