<i>Ficus rubiginosa</i> (the Rusty Fig; Moraceae) provides a keystone food resource for a diverse array of vertebrate frugivores in eastern Australia. These frugivores, in turn, provide vital seed-dispersal services to the fig. The aims of this study were to investigate impacts of population size and climatic variation on avian-frugivore visitation to <i>F. rubiginosa</i> at the extreme western, drier margin of the species' range. Eighty-two bird species visited <i>F. rubiginosa</i> trees in this three-year study. Twenty-nine species were frugivores or omnivorous frugivore/insectivores. The number of ripe fruit in a tree had the greatest positive influence on frugivore visitation (p < 0.0001). Fig-population size influenced the assemblage of frugivore species visiting trees but not the number of frugivores or the rate of frugivore visitation. The number of ripe fruit in a tree was negatively associated with declines in rainfall, to total losses of standing crops through dieback and lack of crop initiation. Predicted long-term declines in rainfall across this region of eastern Australia and increased incidence of drought will lead to reduced crop sizes in <i>F. rubiginosa</i> and likely reduce the viability of local populations of this keystone fig. This will threaten the mutualism between <i>F. rubiginosa</i> and frugivores across the region.