As a human, counting outs is easy. Usually I find myself doing it to get my chances to have a flush, a straight, a three of a kind, etc... (with the multiply by 4 and multiply by 2 rule). I then assume that if I have that, I win, if not I lose.

I'm wondering how to put this in a sort of algorithmic form...

The problem I'm having is, WHAT outs should be counted? I know as a human player, I don't count outs for possible pairs. If I have a 3 and a 4 in my hand, and there's a 5 and a 6 on the table, then I find myself considering 2's and 7's as outs. I don't really take the possibility of a pair of 3's or 4's into account.

Do you think making an AI count outs could be helpful, and if so, which outs should be considered, which not, and how?

I think a bot working with counting outs would probably not be as accurate as one using monte carlo simulations to calculate win chances, but it could be a lot faster (and maybe a bit human-like?).

Why not make the AI count outs as you just explained you yourself do? Does it have to be more complicated?

I personally don't find counting outs to be as helpful as calculating your equity against an opponent's range of hands via Monte Carlo. The hard part is assigning a reasonably accurate range of hands to your opponent. Also, I wouldn't worry about the speed of Monte Carlo simulations being a problem.