Using Newell's comment 9 above, I did some testing on our blade chassis with the node I initially saw this on.

To investigate this, I logged into the MAAS server and re-commissioned the node.  I noted the IP address used during commissioning and connected to the node and did a 'tail -f' on the cloud-init log to watch the commissioning process.

Once Commissioning was completed, it looks as though the "power down" command issued in the running OS, was successful.  I was force logged out and the machine became unresponsive to SSH requests.

Now, at this point, there are three things:
1:  MAAS Shows the power state as ON, but overall state as Ready.
2:  The API is returning ON:
>>> power_state_ucsm(url, username, password, uuid)
up
'on'
3:  UCSM UI shows power state as OFF.

So it seems the API is telling MAAS the power is still on, although the UCSM UI shows it as off.

Now... in order to resolve this issue, I must manually use the "Power Off" action from the MAAS Action menu for this node.  This tells MAAS to send a Power Off command via UCSM to the node. This causes the API to finally tell us that the node is OFF.

This is the XML from the API after commissioning:
<configResolveChildren
cookie="1471266515/8e269db3-c7ff-4b26-9c13-addd9668db2c"
response="yes" classId="lsPower"> <outConfigs>  <lsPower
dn="org-root/ls-B200M3_SND/power" state="up"/> </outConfigs>
</configResolveChildren>

Note, the node appears to be powered off at this time according to the UCSM UI and the unresponsiveness of the node to pings and SSH, but the API says it's still up.

After manually powering the node down, the API shows the state as Down, correctly.
<configResolveChildren
cookie="1471267730/436439b8-378d-4cc8-bfd6-fcf7bc5f1ca0"
response="yes" classId="lsPower"> <outConfigs>  <lsPower
dn="org-root/ls-B200M3_SND/power" state="down"/> </outConfigs>
</configResolveChildren>

Now if I deploy the system, once I've gotten the correct power state from the API, the node powers on, and the API and UI reflect this:
<configResolveChildren
cookie="1471267817/220ac2c0-c44d-4179-871b-09bbdb347c6b"
response="yes" classId="lsPower"> <outConfigs>  <lsPower
dn="org-root/ls-B200M3_SND/power" state="up"/> </outConfigs>
</configResolveChildren>

Once deployed, I have two Action options in the MAAS UI.  I can Power Down or Release. If I Power Down, the state changes correctly.  If I release, it ALSO changes correctly.  I think this is because when I use either of those, we DIRECTLY issue a Power Down via the API.

But if I issue a power-down using "shutdown -t now" from inside the OS on the node, the API incorrectly reports the node as being powered up, and thus MAAS incorrectly reports the node as being powered up.  The USCM applet correctly shows the power as Down.

So my suspicion here is that when the node is powered down in OS, using the "shutdown" command in Ubuntu, the node DOES power off, but the API is never updated somehow.  So it seems like it's very possible to get the API out of sync with reality by simply using in-band poweroff commands.

I also left the node sitting for 20 minutes after I issued the in-OS shutdown command and the API still said that the node was powered on, so whatever mechanism updates the tables the API is getting it's info from is not probing the nodes for current status, or at least that's the impression I get.

Note, I am NOT able to launch the KVM Manager to see the actual direct-attached console.  Neither Newell nor myself could get that to work because of Java issues (who knows what the working combo of Browser version, Java version and web plugin version are).  I CAN get the UCS Manager applet to load.