Antigens on the surface of pathogens activate the primary response. When you're infected a second time with the same pathogen which has the same antigens on its surface they activate the secondary response and you don't get ill
However some pathogens can change their surface antigens - antigenic variation (different antigens are formed due to changes in the genes of a pathogen)
This means that when you are infected for a second time, the memory cells produced from the first infection will not recognise the different antigens - the immune system has to start from scratch and carry out a primary response again to these new antigens. This primary response takes time to get rid of the infection - get ill again
Antigenic variation makes it very difficult to develop vaccines against some pathogens for the same reason. Examples of pathogens that show antigenic variation include HIV and the influenza virus
The influenza vaccine changes every year - antigens on the surface of influenza virus change regularly, forming new strains
Memory cells produced from vaccination with one strain will not recognise other strains with different antigens - strains are immunologically distinct.