From the early days of nuclear-structure physics, the nature of the second 0+ state in $^{16}$O has been an attracting issue. This nucleus is a typical doubly magic nucleus, and its second 0+ state is interpreted as a four-particle four-hole state. The excitation energy of this state, however, looks too low, considering the large shell shell gap that is naively estimated to be about 11 MeV. In fact, the previous large-scale shell-model calculations (Haxton and Johnson; Warburton et al.) assumed a quenched shell gap from this value. One of the primary aim of the present srudy is how the use of the narrow gap is justified. We focus on the role of correlation energy due to particle-hole excitation, and find that the actual shell gap must be narrower owing to correlation energy compared to a simple estimate from the separation energies. We then achieve a consistent description of the "shell gap" and deformed states around $^{16}$O. The same idea has recently been applied to the $^{40}$Ca region. The N=Z=20 shell gap is determined in the same way, and the resulting shell-model Hamiltonian well reproduces four-particle four-hole bands that appear systematically around $^{40}$Ca.