General-purpose machine-learning force fields (MLFFs) for materials are now trained on the order of 100 million structures, assuming transferability requires such scale. We test this premise on OMat24, the largest available materials dataset, by training MACE models on subsets of 6.5 million atoms, less than 1% of OMat24, selected in three complementary ways: at random, by latent-space clustering of a pretrained MLFF (DIRECT), and by a new model-free method maximizing the diversity of discrete, interpretable local environments (EnvTok). All three models approach the full-data reference MACE-OMAT-0 model to within a few tens of meV/mathringA in forces, and remain close to or exceed it on formation energies, elastic moduli, heat capacities, Li⁺ migration barriers, and molecular-dynamics stability up to 3000 K and 500 GPa. This near-independence from the selection strategy shows that chemical environment diversity, not size, defines OMat24's information content: the remaining 99% of structures are largely redundant. Broad sampling of chemical space thus remains essential, but reference labels are needed only for structurally and chemically distinct configurations. Using atomic positions and species, optionally augmented by magnetic moments, EnvTok identifies these before any labelling or training, yielding compact, architecture-independent datasets. This hundredfold reduction makes hybrid-DFT or higher-level relabelling of general-purpose training sets feasible, and frees computational budget for extending environmental coverage and multi-fidelity learning.