“I didn’t have the best of memory those days” because, you know, it was established that Walky’s brain was mindwiped like a zillion times.  Enough for him to eventually be immune to it, as was important once.