Wednesday, July 13, 2011

Lost Animation when loading Referenced Rigs

This is Particularly relevant to referenced Rigs which have characterSets.

So this has come up a few times on the forums and I've also had emails asking how to fix it so thought I'd drop some info up here. The issue is that at some point the referenceEdits that connect Animation Data to a referenced rig can get broken or corrupted. This means you can be animating away all day, saving incrementals as you go, completely unaware that when you load this scene in, all animation will be lost..or so it would seem.

see this post recently:
http://forums.cgsociety.org/showthread.php?f=7&t=990314

The issue is usually the referenceEdits themselves. What sometimes happens is that the 'connectAttr' block in the referenceEdits gets blanked. We have no idea why, neither does Autodesk, it's completely random. Its also more common on files that came from 2010 and have been loaded in 2011. It also was an issue with rigs that had characterSets on the early release of 2011 IF you were using AnimLayers. (pre hotfix2)

So what to look for?

If you open up the Reference Editor then go to file>referenceEdits. In there you should get a list of all the edits performed since that file was initially referenced in. It's basically a macro that the file load uses to reconstruct the file. So, take a look in the list, see if you still have a large 'connectAttr' block. These should be the connections from the characterSet (placeholder list) to the anim curves themselves. If these edits get corrupted then as I said, you can spend all day saving incremental saves which are all broken but you wouldn't have been aware of them until you reloaded the scenes and the reference list got re-passed.

Now there are some good things to help you. Firstly the fact that the connections aren't there doesn't mean that there's no link from the anim curves to the reference. What happens is that the curves will be left connected to the references RN reference node. If you graph the referenceNode in the hypershade you should still see the connected anim data. It's what happens when you unload a reference, the anim data is cast back to the referenceNode for safe keeping.

So, graph the data, see if it's there, look at the referenceEdit list, see if it is correct or not. If you have no 'ConnectAttrs' then its the same issues we have.

Fixing it is a different issue. Basically we need to write a little tool which spins through the characterSet plugs, then the animCurves left connected to the RN node, does a nameMatch, then reconnects the data. The following dropped into a Python script Tab should do it, just select the characterSet you want reconnected!

import pymel.core as pm
import maya.cmds as cmds
cSet,type=pm.ls(sl=True,st=True)
refNode=cSet.referenceFile().refNode

if not type=='character':
   raise StandardError('You must select a CharacterSet to reconnect')
if not refNode:
   raise StandardError('Given characterSet is not from a referenced file')

animCurves=refNode.listConnections(type='animCurve',s=True)
cSetPlugs=pm.aliasAttr(cSet,q=True)

for plug in cSetPlugs[::2]:
   for anim in animCurves:
       if anim.split(':')[-1].endswith(plug):
           print '%s >> %s' % (anim,plug)
           pm.connectAttr('%s.output' % anim,'%s.%s' % (cSet,plug),force=True)

Simple but effective fix to get round the bug. I think in light of the amount of people posting things like this I'll follow this up with the Maya dev team again. We get this maybe once a month if that, actually probably less than that now we're all on 2011SAP release.

hope that helps folks. I know, referencing is like a black art, god knows I had enough help 7 or 8 years ago when we first started with referenced pipelines.... so thought I'd share the Bug work around.


Mark

53 comments:

  1. Cheers for this post on this damn annoying bug!

    Its happened a few times here, initially I wasn't skilled enough (at all) to write a more generic script to cover all the rigs (which was on my list to do!) to reconnect the attrs to the anim graphs.

    cheers again!

    ReplyDelete
  2. Great post Mark! I was going crazy trying to figure this one out. I ran your script but I get the following error:

    # Error: AttributeError: 'NoneType' object has no attribute 'refNode' #

    The character is definitely referenced so I'm not sure what I'm doing wrong. I'm also starting to wonder about the reliability of .ma files. The last studio I worked for saved exclusively in .mb format and they never had this particular issue.

    Anyhow, thanks again and keep up the great work!

    ReplyDelete
  3. That sounds like the node thats selected can't be resolved as a reference (line 4 in the script above). You sure you selected the CharacterSet and that the set is from the reference file and not generated in the current scene? Try the first 4 lines on another referenced node and see it that passes. Also don't try this on subCharacterSets, apart from anything else it's because subSets are crap!

    ReplyDelete
  4. Hmmm, that might be my problem. I created the character set for the referenced character in the current scene. This is what I did all the times I lost animation. I tried running the first 4 lines but it seems to always crash maya.

    Ah well, seems like the best option for me might be to avoid character sets on referenced characters altogether! :)

    Anyhow thanks again for the tips!

    ReplyDelete
  5. Not a good idea, CharacterSets kind of change how the data is handled on loading so really I'd always make the chSets in your original file prior to referencing. If you make them in the file once referenced then you're adding all that data to the ReferenceEditList which can be a bit flaky anyway. That said the script above could still work with a few lines of modification. Select the RefNode on a broken scene and graph it, if the refNode still has the animationCurves connected then let me know.

    ReplyDelete
  6. Hey Mark, really appreciate this help.

    I'm having a problem just like this (from the cgtalk thread), yet I'm not using any character sets. Would there be a way to modify the script to work in that situation? Or am I facing another issue?

    ReplyDelete
  7. This issue we think, is in the Maya ascii writer itself, the dev team have rolled it back to a version from 2009 to by-pass the issue. Maya2012 SP1 (out soon) I'm hoping will have the bug fix in it.

    In your case the problem with the script is that it needs some whay to know what it's meant to be connecting and to what. So characterSets work well as they're a list of attributes which you know the anim data should link upto. If the animation curves are hanging off the ReferenceNode but not connected to the your controllers then you've hit the same issue, corrupted refEdits.

    Running a fix is easy enough, but the code needs to know what controllers and what attrs on those controllers need connecting, without a set it's a lot harder. Do you have a selection set for your rig?

    ReplyDelete
  8. Mark, first thanks for the post. You just saved me a lot of pain. One change I needed to make to the code was flipping the splitting of the namespace. E.g.

    if plug.split(':')[-1].endswith( anim.name( ) )

    All my plugs seemed to have the reference namespace, while my anim curves were a mix of some that did and some that didn't.

    character set plug name:
    BOB:RG:CTL:world_position_rotateZ

    anim curve name:
    world_position_rotateZ

    Does the reversing of the code make sense to you? I guess if the character set is referenced then the the plugs should have the extra namespace in front.

    ReplyDelete
  9. I guess rather than flip it on it's head really you could just strip the namespace from both ends to make sure you give it the best possibly chance of matching.

    if anim.split(':')[-1].endswith(plug.split(':')[-1])

    If the plugs have namespaces then did you make the characterSet after you'd referenced the rig in?

    ReplyDelete
  10. Sir, you are brilliant.

    Saved me a lot of trouble.

    ReplyDelete
  11. "Michael: I'm having a problem just like this (from the cgtalk thread), yet I'm not using any character sets. Would there be a way to modify the script to work in that situation? Or am I facing another issue?"

    I'm having the same problem, I'm using a file given to me by my school, and I don't think it has a Character Set. I do however have a Mel tab to select all my controllers during animation. Does this help me any?

    I'm pretty new to the Script Editor at all so not much of this makes much sense.

    ReplyDelete
  12. i don't have any connected attr. i don't have character set. and i try to paste the script you gave in phyton and play it . this is what i got.. # Error: IndentationError: unexpected indent #
    i'm using referenced andy rig and maya 2012.. thx..

    ReplyDelete
  13. brilliant. i don't believe it, the world is so huge and so many brilliant artist, i found one. thanks for helping me, mark!

    ReplyDelete
  14. hello Mark

    # Error: IndentationError: unexpected indent #

    same error as " yogayasahardja"
    can you help me mark?

    ReplyDelete
  15. If you're having indentation issues when copying the code to a Python tab then use the 'view plain' option in the syntax block (at the top). That'll strip the unwanted tabs off and give you the original code syntax. Copy and paste that into the Scripts Editor instead and it should work.

    Mark

    ReplyDelete
  16. it works !!

    i juste select the wrong caractere ...my mistake
    thanks mark !

    ReplyDelete
  17. hello Mark and everybody,

    I m Samuel from holland, i have acounter the same problem everybody seems to be having and run the script to fix the problem. But now I m having an problem from running the script import pymel.core as pm
    import maya.cmds as cmds
    cSet,type=pm.ls(sl=True,st=True)
    refNode=cSet.referenceFile().refNode

    if not type=='character':
    raise StandardError('You must select a CharacterSet to reconnect')
    if not refNode:
    raise StandardError('Given characterSet is not from a referenced file')

    animCurves=refNode.listConnections(type='animCurve',s=True)
    cSetPlugs=pm.aliasAttr(cSet,q=True)

    for plug in cSetPlugs[::2]:
    for anim in animCurves:
    if anim.split(':')[-1].endswith(plug):
    print '%s >> %s' % (anim,plug)
    pm.connectAttr('%s.output' % anim,'%s.%s' % (cSet,plug),force=True)
    # Error: ValueError: need more than 0 values to unpack #
    in maya 2012.
    I m new python so i dont really know what to do...

    do anybody know how to fix this problem...pls HELP

    ReplyDelete
  18. You may find that the referenceNode has no animation curves connected to it, in which case you'd get this error. I'd suggest selecting the refRN node itself and graphing it in the hypergraph, see if it has animation curves connected to it in the first place.

    If not then this approach to reconnecting wouldn't work, there are other ways round it however. You could pass in all animation curves in the scene rather than those connected, a little bit brute force but if the refNode has no connections it's the only option open to you.

    We've used both methods around the studio, glad to say that since 2012 we've not had this issue at all thank god!

    ReplyDelete
  19. thank u very much Mark my animation is back

    ReplyDelete
  20. This comment has been removed by the author.

    ReplyDelete
  21. Peter, the code above runs just fine on the file you sent me BUT, the Andy rig has subCharacters and you need to reconnect each of those individually. If you simply run the code on the top level "Andy_All" characterSet nothing will happen. Hope that helps

    ReplyDelete
  22. So is everyone thats still needing this fix running older version of Maya the bug was meant to have been fixed in 2012SAP?

    ReplyDelete
  23. Hi guys, I want to ask a question.
    I change some control objects, from one subCharacter set to anothe one.
    For example:
    I create a character set name Amy_CharacterSet
    With the sub characters, upperBody, leftArm, rightArm.

    Then I reference the file in a new scene, and start animating. Then I go back to the original file, the one I reference, and delete the leftArm and rightArm subCharacterSets, and add these controls to the upperBody subCharacter.
    When I go back to the animation file, the animation of the left and right arms where lost!
    Is there a way to do this? change the subCharacters sets, go back to the animation file with out lossing the animation?

    By the way, your script help me recover the animation data, What I did was create again the original subCharacters, and adding the control objects to the subcharacters, like before the change, and in the animation file running your script in the subcharacters i have lost the animation.

    ReplyDelete
    Replies
    1. I just test the script on the new characters sets, and it worked!
      With this script I can go back, change the name of my characters sets, create, delete, etc, sub characters sets, and with the script all the animation gets link to the proper attribute! AMAZING!

      Delete
  24. Glad it worked for you but I really, seriously wouldn't get into the habit of changing CharacterSets on referenced data. The way Maya handles cSets under the hood during file load on referencing is very complex. Unlike normal nodes none of the data is hooked up by dagPath, it all goes via a placeholder list which in turn points to 3 sublists that hold the indexes for angular, linear and unitless type animCurve data. So whilst this code will reconnect, you have to bear in mind that these reconnections are all then being added back into the referenceEdit list for that scene.

    To be honest we ditched sub-characterSets years ago as they're bloody useless when it comes to handling data in Trax.

    ReplyDelete
  25. Hello Mark. I've encountered this problem too. But unfortunately I'm still new to maya (less than 1 month), and even virgin on maya programming. Case in point, when you said:

    "If you open up the Reference Editor then go to file>referenceEdits. In there you should get a list of all the edits performed since that file was initially referenced in. It's basically a macro that the file load uses to reconstruct the file. So, take a look in the list, see if you still have a large 'connectAttr' block. These should be the connections from the characterSet (placeholder list) to the anim curves themselves."

    I even don't know which part in the Reference Editor that said 'connectAttr' block, let alone seeing the connections. I'm that noob.

    So, maybe you could tell me step by step for dummies like me, from how to put the script, run it, and then do what?

    This is my understanding of how to use (which I'm sure is way of base):
    1. Open the .ma that contain my animation
    2. run the dkAnim()
    (2a. should I select something here?)
    3. save file .dkanim
    4. And then what?

    There are options for "search for", "replace with", "add prefix", what are these for? I don't even know what to search.

    Thanks

    ReplyDelete
  26. So you have a file that has a characterSet which is referenced and the animation has disappeared on file load right? Ignore the refEditor is you're not familiar with it.

    Just copy and paste the above script into the ScriptEditor (make sure you're on the Python tab and make sure that the Tabs/spacing in the script are maintained as in Python Tab indentations are very important). Then select the characterSet and Execute the script. (CommandMenu in the scriptEditor>Execute)

    It should, if its the same bug, re-hook the animation data backup.

    ReplyDelete
  27. Sorry, for some reason this post:

    http://blog.ryantan.net/2012/01/recovering-lost-animation-on-referenced-charactersets-in-maya-2011/

    Made me think you're the one who make dkAnim(). Sorry for my stupidity!

    Anyway, I tried that, and it produced error like this:

    # pymel.core : Updating pymel with pre-loaded plugins: OpenEXRLoader, DirectConnect, mayaHIK, AbcImport, ikSpringSolver, tiffFloatReader, AbcExport, objExport, VectorRender, quatNodes, mayaCharacterization, gpuCache, rotateHelper, Substance, MayaMuscle, OneClick, AutodeskPacketFile, retargeterNodes, fbxmaya, matrixNodes, stereoCamera, ik2Bsolver #
    # Error: line 1: StandardError: file line 7: You must select a CharacterSet to reconnect #

    Umm... Another noob question, when you said 'characterSet', it means the referenced object (that lost the animation) right? Or it's different?

    ReplyDelete
  28. i don't have any connected attr. i don't have character set. pasted the script you gave in phyton and play it . this is what i got.. # Error: ValueError: too many values to unpack #

    Also I dont know were to find the refNode and graph it in the Hypershade any Tips?

    ReplyDelete
    Replies
    1. I found the Node! and I have my Animation curves connected :)
      but Im still getting the

      # Error: ValueError: too many values to unpack #

      Delete
    2. What did you have selected when you ran the code? I must admit I've been thinking about doing a different version of this that doesn't rely on ChSet, instead connects blindly based on animCurves

      Delete
    3. I have the exact same problem over here :(

      Delete
  29. I had selected all my character`s animation controls.

    I remember that some attributes didn’t have any keys on them before the bug, can that affect the script?

    Thanks man, awesome blog by the way!

    ReplyDelete
  30. The script above is designed purely for running from a characterSet selection and rewiring data to it, from the found referenceNode. Selection for it should only be the characterSet.

    Now if you're getting the same bug in referenced scenes where you have no characterSet then it could be modified, BUT if you are also running animation layers it's way, way more difficult to reconstruct the data. I can certainly push out a version that deals with the simple reconnect to the selected nodes, if matching animData is found.

    ReplyDelete
    Replies
    1. Grate! luckily I’m not using anim Layers. Thanks for your time Mark. I’ll be sharing your blog with every one; awesome stuff!

      Delete
    2. Don't suppose you have a file that's failed? I'm wondering if in this case the animCurves are still connected to the refNode?

      Delete
  31. Check the new Lost Animation Part 2 post ;)

    ReplyDelete
    Replies
    1. Problem 100% Fixed!

      Lost Animation Part 2:

      http://markj3d.blogspot.com/2012/09/lost-animation-part2.html?showComment=1346961028813#c7343634541055468561

      Delete
  32. Hi Mark,

    Ive ran into a problem not really covered on this blog, but i have exhausted all other resources...

    Ive opened my scene that has a number of referenced props, environments and a character on which i have been animating.
    After closing this file and then reopening it, it is as if the referenced character never excited in the first place as it isn't coming into the scene and not even showing up in the reference editor. Have I lost this animation all together or is this salvageable?

    Thanks.

    ReplyDelete
    Replies
    1. must admit this rings bells. Firstly are you saving as .ma's? there was a bug in the ascii writer in the early releases of 2012 which corrupted the file format itself from time to time. If the reference node itself isn't showing up in the file then take a look in the outliner, turn off display>dag objects only and see if the animation curves themselves loaded up. If they did then there's a chance you could save it. Might be worth trying to rereference in your rig (with the same namespace as before) and run the code in the Lost Anim Part2 post of mine.

      Failing that you could always mail me the file, happy to take a look.

      Delete
  33. just want to say- thank you very much

    ReplyDelete
  34. Thank you so much for this. Worked like a charm, you saved me a week of work!

    ReplyDelete
  35. I've had no luck with this script, as I don't have any character sets. I've been looking on various forums to try reconnect the anim curves to the rig, but no luck. I do have blocking from before my lost animation saved out in a studio library anim file - so a workflow I found suggested create a new reference and to be safe, save out the animation before closing the file, every time. That way if the the animation isn't connected to the rig upon reopening, you can put it back. I'm wondering if there's another way to reconnect the anim nodes if you don't use character sets??? Thanks!

    ReplyDelete
  36. Waaah, I really need to recover my animation! I tried doing the steps that Mark posted but somehow it doesn't work, I'm sure I'm doing something wrong but I don't know where! I get an error saying....# Error: ValueError: file line 3: need more than 0 values to unpack #

    ReplyDelete
  37. Ok to whom it may concern, I couldn't make this python script to work for me so I tried doing this which amazingly worked and saved me a few days of work!

    Open your .ma scene file in text editor or word pad
    at the upper right corner, select Replace
    Find what: nan(ind)
    Replace with: 0
    Then click on Replace All

    You can do it again by now just typing "nan" in Find what then replace with 0 and Replace All again just to be sure.
    Then save your file. (Make sure you overwrite it and not save as)
    Also a good idea to make a copy of the original .ma file before doing this just in case.

    Then now you can open the new modified .ma scene file in Maya.

    It worked for me, hope it works for you too!


    Replace all "nan" or "nan(ind)" with "0".

    ReplyDelete
  38. Hi Mark,

    This sounds like a wonderful solution to my problem!
    Unfortunately the referenced rig that I used doesn't seem to have character sets.

    I'm not sure what you mean by: "graph the referenceNode in the hypershade"
    I assume to click the malfunctioning control in the "List Reference Edits". I saw keys connected to this before, but I'm not sure anymore as they don't seem to show up after trying different fixes for a while.

    I created a character set for the torso, but as I run the script I get the following error:

    # Error: ValueError: file line 3: too many values to unpack #

    What am I doing wrong here?

    Thanks for your help!


    ReplyDelete
    Replies
    1. i am also curious about this! i don't use character sets either

      Delete
  39. wow is this still happening! If you have a file that's broken and can share by all means drop me a line. The reconnect was written YEARS ago and I can't believe how many people have found this post over the years.

    ReplyDelete
  40. still an issue in 2023, I was using a Mgear rig with no characterset, just selection sets

    ReplyDelete
    Replies
    1. Did you ever find a fix? Im having the same issue

      Delete
  41. Hi, I have a same issue using Maya 2023 and I really don't know how to fix it.

    ReplyDelete
  42. Animation rigging  requires both technical expertise and a deep understanding of anatomy and physics.

    ReplyDelete