Basically, the source code is turned into a grid of characters. The "instruction pointer" moves over these characters, doing whatever the character indicates. " toggles "string mode" which just pushes the ASCII values of the characters to the stack. After each instruction, the IP just moves one character in the direction it's facing. The instructions ^v>< are read as arrows, and change the direction that the IP moves.

"Comments" are just parts of the code that the IP doesn't traverse.

You construct loops by having the IP actually move around in a loop:

>"a"v
^ , <

here, "a" is always approached from left to right, so it pushes 97 to the stack. , pops from the stack and prints that number as a char, which in this case will always be a.

v <
>","v
  ^<

The code here moves in a figure-8. It first goes down, right, and across ",", pushing 44 to the stack. it then goes down, left, up, and across ,, printing char(44) - ,. Then, it moves left to the starting position and repeats.

You do branching with | and _, which pop from the stack and move down/right if it's 0, or up/left if it's not.

If the stack is empty, all pop operations give 0.

So what that hello world does is go LEFT, right at the start, which loops around and crosses "Hello, world!" in reverse, pushing them to the stack in reverse order. Then, it jumps across the semicolons, and >>#; reverses its direction. Then it runs through >:#,_@, which prints the top value of the stack, or exits if it pops 0, which prints the whole string in reverse-reverse order.