Let me try again ... LOL .. this is a nice challenge.

Present time Bran - leaving cave: Hears Meera talking to Hodor - the semi-conscious Bran recognizes this although he is not fully awake and thinks of the concept/person Hodor - this is one instance of Hodor occurring within Bran's mind

Past visiting Bran - at Winterfell: See's Wyllis and thinks of him like "Hey, there's Hodor my buddy!" - this is a co-occurring instance of Hodor within Bran's mind

Bran is super-green-seer -- with amazing abilities - abilities allowing him to enter into the minds of animals and humans and control actions and movements - essentially setting aside what is already going on in that target mind and taking over. His mind allows him to project himself across time and space - so he simultaneously had the image of Hodor in his mind in both present time and past time - Hodor being one of his target warg animals - I guess warg's have special relationships with their target animals, as we've seen with Orvel the warg when Sixskins warged the eagle and still felt Orvel's presence - so maybe there is not only a future residual warg memory, but this scene might explain and identify a past residual warg memory - it could be that through time specific men and specific animals have been matched up through more than just happenchance -- Wyllis was meant for Bran, just as Ghost was meant for Jon, or Nymmera was meant for Arya -- the difference being that Bran has greater powers and more warging mates -- heck I don't know -- it's a cunnundrum!

Side note red herring divert attention away from this mess: Obviously it should be understood that Littlefinger has very special skills also - skinchanger or whatever - he's form Iron Bank blood, but uses a mockingbird as his animal identity