Lost Animation Part3 !!

Well it seems this bug is very much still in Maya 2013 and for some reason we seem to be hitting it with more frequency at work. For those of you not aware this it's a bug where the referenceEdits in Maya don't get reconstructed correctly and you end up opening Maya scenes with characters in T-pose and all the anim data thrown back to the referenceNode.

Now over the years I've posted 2 options for reconnecting this data and had SOOO many emails about it, but now that we all run animLayers reconstructing this lost data is a damn sight harder. I'm just wondering if anybody else out there had any ideas of why animation keeps disconnecting itself from referenced rigs with chSets. I've been working past this bug for nearly 7 years now and thought it was quashed!

 Back to Autodesk I guess.

 Mark

Lost Animation - Part2!

God how many times have I been asked for this since posting the 'Lost Animation' thread nearly a year ago now. So here's a complete and utter dirty hack for those who have requested it. Unlike the other fix code this is completely blind. You select the controllers that are no longer connected to the animation data that they should be, and this, in an ideal world, will reconnect the required animCurves for you.

There are a few HUGE expectations here, mainly that the scene is clean and that the animCurves are still consistently named against the channel they were keyed against. As I said, this is a hack but it seems to be what a lot of people are in need of.

So the expectation is that an attr on a controller called NameSpace:L_Foot.translateX should be connected to NameSpace:L_Foot_translateX, or L_Foot_translateX depending on the flag stripNamespace thats in the code.

As I said, this is a really quick hack, but I figured what the hell. Oh and if the data went through AnimLayers....sorry, you're stuffed!

Mark

import maya.cmds as cmds
nodes=cmds.ls(sl=True,l=True)
chns=[]

#Change this to False if the curves are not in the rootNamespace but
#in the sameNamespace as the controllers.
stripNamespace=True

#build up the main lists
animCurves=cmds.ls(type='animCurve',s=True)
[chns.extend(cmds.listAnimatable(node)) for node in nodes]    
    
for chn in chns:
    if stripNamespace:
        animCurveExpected=chn.split(':')[-1].split('|')[-1].replace('.','_')
    else:
        animCurveExpected=chn.split('|')[-1].replace('.','_')
    if animCurveExpected in animCurves:
        if not cmds.isConnected('%s.output' % animCurveExpected,chn):
            print '%s >> %s' % (animCurveExpected,chn)
            cmds.connectAttr('%s.output' % animCurveExpected,chn,force=True)

ReferenceQuery Madness!

If you're managing references in Maya and coding functions around it you'll know that the reference commands and query commands available to you are very limited. But here's a kicker that I found yesterday that to me, makes absolutely no sense what-so-ever!

So we all know how flaky the referenceEdit list is, but here's a bit of logic for you that we didn't realize until one of our tools started playing around.

Simple test, make a sphere in a fresh scene and save it, then reference it back into a new scene. Now make a cube and parentConstraint the referenced sphere to it. Now look at the referenceEdits, you should see this:

All well and good. So you'd think that if you queried the refEdits that's exactly what you'd get right? No

cmds.referenceQuery('SphereRN', successfulEdits=True, failedEdits=False, es=True) 

#Result:
[u'parent -s -r "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1" "|Sphere:pSphere1"',
 u'connectAttr "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintTranslateX" "|Sphere:pSphere1.translateX"',
 u'connectAttr "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintTranslateY" "|Sphere:pSphere1.translateY"',
 u'connectAttr "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintTranslateZ" "|Sphere:pSphere1.translateZ"',
 u'connectAttr "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintRotateX" "|Sphere:pSphere1.rotateX"',
 u'connectAttr "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintRotateY" "|Sphere:pSphere1.rotateY"',
 u'connectAttr "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintRotateZ" "|Sphere:pSphere1.rotateZ"',
 u'connectAttr "|Sphere:pSphere1.rotateOrder" "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintRotateOrder"',
 u'connectAttr "|Sphere:pSphere1.parentInverseMatrix" "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintParentInverseMatrix"',
 u'connectAttr "|Sphere:pSphere1.rotatePivot" "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintRotatePivot"',
 u'connectAttr "|Sphere:pSphere1.rotatePivotTranslate" "|SphereRNfosterParent1|Sphere:pSphere1_parentConstraint1.constraintRotateTranslate"']

So when you use the default query calls you actually get the edits back as if the refNode was in an unloaded state, note the connections to the fosterParent which would get created if I actually did unload the reference! Why, why the hell would you want the default option to return the data in an format that was currently invalid.

So there is a way round it, you use the 'liveEdits' flag. All good right, NO, the liveEdit flag returns the data WITHOUT THE DAG PATH! Short names, who the hell returns short names in any code blocks.

cmds.referenceQuery('SphereRN', successfulEdits=True, failedEdits=False, es=True, le=True)

Rant over, gotta go find a way round this now!

Mark

Lost Animation when loading Referenced Rigs

This is Particularly relevant to referenced Rigs which have characterSets.

So this has come up a few times on the forums and I've also had emails asking how to fix it so thought I'd drop some info up here. The issue is that at some point the referenceEdits that connect Animation Data to a referenced rig can get broken or corrupted. This means you can be animating away all day, saving incrementals as you go, completely unaware that when you load this scene in, all animation will be lost..or so it would seem.

see this post recently:
http://forums.cgsociety.org/showthread.php?f=7&t=990314

The issue is usually the referenceEdits themselves. What sometimes happens is that the 'connectAttr' block in the referenceEdits gets blanked. We have no idea why, neither does Autodesk, it's completely random. Its also more common on files that came from 2010 and have been loaded in 2011. It also was an issue with rigs that had characterSets on the early release of 2011 IF you were using AnimLayers. (pre hotfix2)

So what to look for?

If you open up the Reference Editor then go to file>referenceEdits. In there you should get a list of all the edits performed since that file was initially referenced in. It's basically a macro that the file load uses to reconstruct the file. So, take a look in the list, see if you still have a large 'connectAttr' block. These should be the connections from the characterSet (placeholder list) to the anim curves themselves. If these edits get corrupted then as I said, you can spend all day saving incremental saves which are all broken but you wouldn't have been aware of them until you reloaded the scenes and the reference list got re-passed.

Now there are some good things to help you. Firstly the fact that the connections aren't there doesn't mean that there's no link from the anim curves to the reference. What happens is that the curves will be left connected to the references RN reference node. If you graph the referenceNode in the hypershade you should still see the connected anim data. It's what happens when you unload a reference, the anim data is cast back to the referenceNode for safe keeping.

So, graph the data, see if it's there, look at the referenceEdit list, see if it is correct or not. If you have no 'ConnectAttrs' then its the same issues we have.

Fixing it is a different issue. Basically we need to write a little tool which spins through the characterSet plugs, then the animCurves left connected to the RN node, does a nameMatch, then reconnects the data. The following dropped into a Python script Tab should do it, just select the characterSet you want reconnected!

import pymel.core as pm
import maya.cmds as cmds
cSet,type=pm.ls(sl=True,st=True)
refNode=cSet.referenceFile().refNode

if not type=='character':
   raise StandardError('You must select a CharacterSet to reconnect')
if not refNode:
   raise StandardError('Given characterSet is not from a referenced file')

animCurves=refNode.listConnections(type='animCurve',s=True)
cSetPlugs=pm.aliasAttr(cSet,q=True)

for plug in cSetPlugs[::2]:
   for anim in animCurves:
       if anim.split(':')[-1].endswith(plug):
           print '%s >> %s' % (anim,plug)
           pm.connectAttr('%s.output' % anim,'%s.%s' % (cSet,plug),force=True)

Simple but effective fix to get round the bug. I think in light of the amount of people posting things like this I'll follow this up with the Maya dev team again. We get this maybe once a month if that, actually probably less than that now we're all on 2011SAP release.

hope that helps folks. I know, referencing is like a black art, god knows I had enough help 7 or 8 years ago when we first started with referenced pipelines.... so thought I'd share the Bug work around.


Mark

Relative Namespaces in Maya2011

I just wanted to add a quick pointer to the namespace -rel flag in Maya 2011 and how it can make managing namespaces a whole lot easier. By default this is set to false, meaning that all namespaces are treated from the root space (":") When you look in the Reference editor what you see as the namespace is the actual ns that the file was imported into.

Lets say you had imported a file 3 times on itself, so ended up with nested namespaces:

THIS:IS:ATEST:Myfile.ma

Now if you look in the ref editor you'll see the namespace as ATEST. If you try and change it it'll say the namespace doesn't exist. This is because your by default in root (":") space. This is where the rel flag can really help, it makes the namespace handling RELATIVE to the current space your in.

So mel:

namespace -set "THIS:IS";
namespace -rel true;
file -e -ns "FIXED" "C:/Test_Rig.ma";
namespace -rel false; 
namespace -set ":";

So here were setting ourself into the namespace THIS:IS, making all namespace actions relative to this, setting the new namespace to FIXED and then returning ourselves to root sapce and tuirning off the rel falg. The result is you've now changed the namespace that it said couldn't be changed, ending up with

THIS:IS:FIXED

Mark

Trax, Character Sets and Referencing

Whilst this stuff is fresh in my mind I wanted to put it down, more for my reference but who knows, it may be useful to others.

How Does Referencing work with Character Sets?
As per previous posts, referencing is nothing more than an import of data from one scene into your current, followed by an edit list applied afterwards. The thing is that the refEdits are all based on full Dag node paths, so if you change the name of a node in the master file, or reparent it, your referencing for that edit fails. Namespaces go some way to eliviate clash issues but other than that, it's all name based. In fact, you really need to think of Maya's scene hieracrhy not as a nested scene graph, but as a flat, dag name based database.

So How Do CharacterSets help?
Unlike normal referencing, all connections to objects that are members of a CharacterSet are based NOT on name, but on the membership index of that set. This means that as long as you carefully maintain the internal order of the chSet, you can rename and regroup at will, the connections for your animations will be maintained.

A Little Deaper:
The CharacterSet in Maya is actually mapped in a totally different way internally to that which the users sees. Rather than using names to pass animations to channels it uses characterMapping and characterAlias's. It's actually split down into 3 main internal arrays,

• Map 0 : unitlessValues : animCurveTU - attrs like user defined and scales
  
• Map 1 : linearValues : animCurveTL - transform data
  
• Map 2 : angularValues : animCurveTA- rotational data
  

in your Master Rig file (.ma) that contains the characterSet you see a block that looks something like this:

setAttr ".rm" -type "characterMapping" 206 "RIG:Shoulders_Ctr.SpineBias"
0 1 "RIG:Shoulders_Ctr.rotateZ" 2 1 "RIG:Shoulders_Ctr.rotateY"
2 2 "RIG:Shoulders_Ctr.rotateX" 2 3 "RIG:Shoulders_Ctr.translateZ"
1 1 "RIG:Shoulders_Ctr.translateY" 1 2 "RIG:Shoulders_Ctr.translateX"
1 3 "RIG:Hips_Ctr.rotateZ" 2 4 "RIG:Hips_Ctr.rotateY"

Here the characterSet has 206 members (channels), "RIG:Shoulders_Ctr.SpineBias 0 1" indictates that this is a unitlessValue (Map 0) and it's unitlessValue[1]. "RIG:Hips_Ctr.rotateZ" 2 4 indicates it's an angularValue (Map 2) and angularValue[4].... and so on....

The channels within the ChSet are mapped to these arrays in the order they're found within the ChSet, but going from bottom to top. It's this mapping thats then used to reConnect channels and animCurves when you load or reference the scene.
In the referenceEdits you'll see the connections from animCurves to these lists, which will give you an idea of just how the data is mapped internally on a real scene.

The key thing here is that because it's all based on order, should you switch your reference to a rig whos chSet mapping is different the whole thing falls apart. Say you had a refEdit :

connectAttr animCurveTL4545 LinearValue[4] 

that connected the translateX of the Wrist to it's animCurve, this edit would still just blindly go and remake this connection, even though LinearValue[4] may now be a completely different object. Now in Maya2011 they try and resolve this issue in a post load script that runs after file load, but's it's better to know about than rely on it.

So the key is, maintain the chSets, order is king.

Reference Edits

One crucial thing with referencing is that the Reference Edit List isn't just a macro that's run in the order you performed the actions. Instead it's grouped and more importantly applied by function types. If you open the refEdit List up you'll see the grouped calls:

: setAttr
: addAttr
: connectAttr
: disconnectAttr
: deleteAttr

To get to the edit List open the referenceEditor then File > List ReferenceEdits

This is a contentious issue and has been bought up on the Autodesk Beta forums many times. In Maya2011 there has been some fixes to this to make it more intelligent. Try the following in 2010 and then 2011.

Make a cube and reference it into a new scene, run the following and save this new scene. Now open the file again...

import maya.cmds as cmds
cube=cmds.ls(sl=True)[0]
cmds.addAttr(cube, ln='Test')
cmds.deleteAttr('%s.Test' % cube)
cmds.addAttr(cube, ln='Test')

So we've added an attribute 'Test', then deleted it, then added it again in a referenced file. Now this looks like a stupid thing to do, but lets say you're using attributes as markers in a function, maybe exportFlags etc. In 2010 if you look at the edits you'll see the addAttr, then under that the deleteAttr.... regardless of the fact that the last action was to add the attribute again. When the scene is saved the new attribute won't be there because the delete was the last thing in the editList. More over, unless you go and remove that deleteAttr from the editList itself, you'll never be able to remake it.

2011 has a fix for this case.

Maya Referencing

I've spent the last month or so updating our Referencing Pipeline in Maya and thought I'd share a few tips in light of changes in Maya 2011. This will probably be a multi-part post... too much for one.

Part1: Background and Workflows - Reference crash course

At work we run a fully referenced animation setup, carefully managed by custom character checkers, loaders and asset management systems all designed to by-pass bugs and limitations in Maya native referencing setups.

Where to begin.... How and what should you reference?

Firstly referencing is designed to make life easier, a change in a source file will automatically ripple into all your working files. So in the case of Character Rigs ideally you'd want to reference the modelGeo into a skin file, skin it, reference that into a rig file, rig it, reference that into an animation and animate it. GREAT, you change the mesh and ......BOOOOOOOM! oops wait all my animations broken.. WTF?. In the real world there are just too many unknowns and possible conflicts to go that far. Also you really want to isolate certain changes from stages of production. So point one:

Avoid nesting your references if you can.

We run a 2 stage reference setup, based around a Rig that carries the skinned master character around with it... basically the output skeleton and geo piggy-back the rig's internal skeleton. This means that the riggers can run rig files that have a reference to the characters skin file - a scene with the skeleton and geo skinned, basically the output file required for the game. What this means is that to generate a new animation rig all that's required is to open up the master rig file, switch the reference to the skin file to the new character and bingo, rig done. This is very fast and ensures consistency in the rig itself. The piggy backing also isolates any small changes in skeleton, making it possible to tweak the characters internal structure if really required without destroying all your animation files. This referenced rig is itself then referenced into a second level where the facial is added.

Now that gives you a working rig, and for a short period we used these files for animation. But referencing is based on macros, referenceEdits, and they're just not stable enough to cope with this level of complexity, it means production gets flaky and that comes back to you, the TD. So before passing the files to the animators dereference it. flatten those references out of the file isolating them from rig production.

Referencing the Rig - Naming conventions:
So now we have a production ready rig, no references in it, and most crucially, all nodes NAMED IDENTICALLY in all rigs.. or at least all the master controllers the animators will touch are all named generically. Why not name the rig nodes to match the characters, why not prefix things in the rig file eg BOND_Wrist_Ctr? You need to understand the way referencing is managed internally. Think of it like this, referencing just imports a scene and runs a macro on it which is NAME DEPENDENT, that's crucial and there are ways to by-pass it, we'll get to those later. So in the case above if you'd animated Bond_Wrist_Ctr and then switched reference to Odjob who's rig file had Odjob_Wrist_Ctr nothing would connect up and you'd be left with a broken scene... not good.

Namespaces:
These are the devils work, but required to make referencing in Maya work. The idea is that the generic rig has everything none specifically named, then when you make the reference you use namespaces to denote who the character is. So you'd end up with BOND:Wrist_Ctr. This is a crucial distinction to prefixing. Namespaces are handled by the reference command/file command, when you switch the reference to a rig all you're doing is pointing it to a new file, replacing the imported file nodes and then rerunning that refEdit macro. The namespace will remain as it was after the switch, but that's easily changed from inside the reference editor itself.

I'll explain the tech side of namespace management in the next part..