Total yeast and mold (TYM) levels in inflorescences of high THC-containing Cannabis sativa (cannabis) are regulated to ensure that medicinal and recreational users, especially those with immunocompromised systems, are not exposed to potentially harmful levels. In North America, the limits imposed range from 1,000–10,000 cfu/g of dried product to 50,000–100,000 cfu/g, depending on the jurisdiction. Factors affecting a build-up of TYM in cannabis inflorescences have not been previously researched. In this study, >2,000 fresh and dried samples were assayed for TYM over a 3-year period (2019–2022) to identify specific factors which can contribute to TYM levels. Greenhouse-grown inflorescences were sampled before and after commercial harvest, homogenized for 30 s, and plated onto potato dextrose agar (PDA) with 140 mg/L streptomycin sulfate. Colony-forming-units (cfu) were rated after 5 days of incubation at 23°C under 10–14 h light. PDA provided more consistent counts of cfu compared to Sabouraud dextrose and tryptic soy agars. The predominant fungal genera identified by PCR of the ITS1-5.8S-ITS2 region of rDNA were Penicillium, Aspergillus, Cladosporium, and Fusarium. In addition, four yeast genera were recovered. In total, 21 species of fungi and yeasts constituted the total cfu present in the inflorescences. The variables that significantly (p < 0.05) increased these TYM levels in inflorescences were: the genotype (strain) grown, presence of leaf litter in the greenhouse, harvesting activity by workers, genotypes with a higher abundance of stigmatic tissues and inflorescence leaves, higher temperature and relative humidity within the inflorescence microclimate, time of year (May–October), method of drying buds after harvest, and inadequate drying of buds. The variables which significantly (p < 0.05) decreased TYM in samples were: genotypes with lower numbers of inflorescence leaves, air circulation achieved by fans during inflorescence maturation, harvesting during November–April, hang-drying of entire inflorescence stems, and drying to a moisture content of 12–14% (water activity of 0.65–0.7) or lower which was inversely correlated with cfu levels. Under these conditions, the majority of dried commercial cannabis samples contained <1,000–5,000 cfu/g. Our findings indicate that TYM in cannabis inflorescences are the result of a dynamic interaction between genotype, environment, and post-harvest handling methods. Some of these factors may be altered by cannabis producers to reduce the potential build-up of these microbes. Among the 21 fungal and yeast species recovered from greenhouse-grown cannabis inflorescences, a few could pose a potential threat to human health, while many do not and they could provide beneficial interactions within the cannabis plant. The currently recommended plating methods onto agar media and enumeration of total cfu are unable to distinguish between these two groups.