You use the Past Perfect or Past Perfect Continuous when you're talking about something which could have happened in the past but which didn't actually happen