2 suggested paths from mechanism to consciousness
Irrationality is a necessary (and perhaps sufficient) condition for consciousness?? Think about it this way. You have a system like Drescher's which compiles statistics to direct its actions in a "world." For consciousness, the most important aspect of such a system is that which
"is defined not so much by its particular set of primitives as by its ways of combining structures to form larger ones, and by its means of abstraction -- its means of forming new units of representation that allows the details of their implementation to be ignored." (Drescher, Made Up Minds, p10)
As I read it, what this means is we get a system which combines its primitive processes into higher-level ones so that you get a "commanding" program with any number of functions, or sub-routines. For non-programmers, this means you could have the lower level processes carrying out the detailed statistical analyses to determine actions, while the so-called commanding program is "unaware" of of those activities. All it needs to operate is a "Yes" or "No" from the sub-routine based on its detailed statistical analysis.
What you end up with is a system, if it is indeed ignoring the primitive functions, that is unaware of its own internal processes and how its actions are determined. If it receives a command, say, to stack 3 blocks that are in its world, the primitive statistical processes are going to run in order to determine the locations of said blocks and make the parts move to execute the command. The higher order process of the system, though, undergoes only the "experience" of receiving the command, finding the blocks, and doing it. If you could ask what it is doing, it would say it was following the order, not that it was carrying out statistical analysis. I believe this is the first step towards conscious machines. There is nothing it is like to be a mechanism which merely compiles statistics and uses it to push buttons on and off. And maybe there is nothing it is like to be a higher-level program which "sees" only the results of such compiling and uses the "Yes" or "No" to push other buttons on and off. But maybe there is something it is like to be a higher-higher-higher ... -level program which pulls vast amounts of different types of input together in one orderly mechanism and is ignorant of the extraordinarily complex underworkings.
Why could this be true? Because it could be true that we are such systems. The complex set of processes that go into pouring myself a bowl of cereal feels simpler by orders of magnitude.