Two-dimensional semiconductors could be used to fabricate ultimately scaled field-effect transistors and more-than-Moore nanoelectronic devices. However, these targets cannot be reached without appropriate gate insulators that are scalable to the nanometre range. Typically used oxides such as SiO₂, Al₂O₃ and HfO₂ are, however, amorphous when scaled, and 2D hexagonal boron nitride exhibits excessive gate leakage currents. Here, we show that epitaxial calcium fluoride (CaF₂), which can form a quasi van der Waals interface with 2D semiconductors, can serve as an ultrathin gate insulator for 2D devices. We fabricate scalable bilayer MoS₂ field-effect transistors with a crystalline CaF₂ insulator of ~2 nm thickness, which corresponds to an equivalent oxide thickness of less than 1 nm. Our devices exhibit low leakage currents and competitive device performance characteristics, including subthreshold swings down to 90 mV dec⁻¹, on/off current ratios up to 10⁷ and a small hysteresis.