Notifications
Clear all

[Closed] bvh based face/lipsync capture import?

my mistake. In that case, it’s brilliant.

So I’ve written up a little script that reads in any bvh file (including the ones exported by the zing thingy) and builds a rig for it and loads the animation. There’s no interface yet and I haven’t given it the ability to map the motion onto an existing max rig, but it’s a start. If people are still interested, I can post it here. Or wait a few weeks till I find the time to make it a little more robust.

A quick question about that Zign thing (without looking at the BVH file): Does this indeed produce 3D (!) mocap data from a single camera?

Thanks!
– MartinB

1 Reply
(@luuksteitner)
Joined: 11 months ago

Posts: 0

Yes, it calculates the 3D movements, but you must keep in mind that it gets less accurate when the face turns away from the camera a lot. For example, it’s hard to see the width of the mouth viewed from the side.

Hi Luuk,
hi everybody

i wrote a script recently, which reads bvh files. I did a few tests and its running fine.

Currently the script assigns the motion to pointhelpers , so the next step is to create the bones and assining the motion to them.
Would be nice if some of you could do some tests also and give me some feedback on how to improve the script.

Greetings from Vienna

Markus 

[ http://www.sb-vs.com ]( http://www.sb-vs.com) 

Edit: here is a screenshot, showing the structure of Luuks BVH file …


    /*
    -------------------------------------------------------------
    --	This script reads the contend of BVH files and 
    --	applies the motion data on pointhelpers
    --  Spheres are used to display joints, lines are
    --	drawn between the different joints
    --	Layers are used to sperate pointhelpers from 
    -- 	lines and spheres
    -- 	31.12.2007 Markus Bliem
    -------------------------------------------------------------
    */
    
    PH =#()
    NoLe=#()
    PHLa=#()
    Noof=#()
    NoCh=#()
    PaNo=#()
    DeNo01=#()
    DeNo02=#()
    PHSi=1
    PHBaMa=#()
    
    FiHei=170
    ToNo=0
    BoNo=0
    Srad = 0.2
    LDi= 0.2
    -------------------------------------------------------------
    --Function Create New Layer
    -------------------------------------------------------------
    
    fn NewLa LaNa =
    (
    	NL = true
    	LaCo = layermanager.count
    	for i = 0 to (LaCo-1) do 
    	(	
    		La = Layermanager.getlayer i
    		print La.name
    		if La.name == LaNa then NL = false
    		
    		)
    	if NL == true do 
    	(
    		La = Layermanager.newLayerfromName LaNa
    		La = Layermanager.getlayer LaCo
    		
    		)
    	)
    
    -------------------------------------------------------------
    --Open File
    -------------------------------------------------------------
    FiNa = getopenfilename caption "Select File"
    BVF = openfile FiNa	
    
    skiptostring BVF "MOTION"
    fpe = filepos BVF
    seek BVF 0
    
    fpo = filepos BVF
    Level = 1 
    NoNum = 0 
    NoDo01 = 0
    NoDo02 = 0
    LhNo=0
    RhNo=0
    ChTo=0
    
    ---------------------------------------------------------------
    -- Draw PointHelper
    ---------------------------------------------------------------
    
    NewLa "BVH_PointHelper"
    
    while fpo < fpe do 
    (
    	
    		rli = readline BVF
    		fpo = filepos BVF
    		tes = filterstring rli "	, " splitEmptyTokens:false
    		chk = tes[1]
    	
    		if chk =="ROOT" do 
    		(
    			col =[0,50,150]
    			NoNum = NoNum+1
    			NoNa= "BVPH_" + tes[2]
 			PH[NoNum] = point name:NoNa size:PHSi wirecolor:col cross:true box:true
    			NoLe[NoNum] = Level
    			PHLa[NoNum] ="C"
    			PaNo[Nonum]=0
    			DeNo01[NoNum] = Nodo01	
    			DeNo02[NoNum] = Nodo02
    			)
    		
    		if chk =="JOINT" do 
    		(
    			col =[0,50,150]
    			NoNum = NoNum+1
    			NoNa= "BVPH_" + tes[2] 
    			colte = findstring NoNa "Left"
    			if colte != undefined then col=[200,0,0]
    			colte = findstring NoNa "Right"
    			if colte != undefined then col=[0,200,0]
 			PH[NoNum] = point name:NoNa size:PHSi wirecolor:col cross:true box:true
    			NoLe[NoNum] = Level
    			PHLa[NoNum] ="C"
 			if (findstring PH[NoNum].name "Left")!= undefined then PHLa[NoNum] = "L"
 			if (findstring PH[NoNum].name "Right")!= undefined then PHLa[NoNum] = "R"
 			if (findstring PH[NoNum].name "Finger")!= undefined then PHLa[NoNum] = "F"
 			if (findstring PH[NoNum].name "RightHand")!= undefined then RhNo=NoNum
 			if (findstring PH[NoNum].name "LeftHand")!= undefined then LhNo=NoNum
 			if (findstring PH[NoNum].name "Chest")!= undefined then ChTo=NoNum
    			)
    		
    	if chk =="End" do 
    	(
    		col =[255,225,5]
    		NoNum = NoNum+1
    		naad =""
    		chna = PH[NoNum-1].name
    		if (findstring chna "Left") != undefined then naad ="Left"
    		if (findstring chna "Right") != undefined then naad ="Right"
    		EsNa = filterstring chna "_"
    		naad = EsNa[2] + "_EndSite"
    		NoNa= "BVPH_" + naad
    		PH[NoNum] = point name:NoNa size:PHSi wirecolor:col cross:true box:true
    		NoLe[NoNum] = Level
    		PHLa[NoNum] ="C"
    		NoCh[NoNum] = 0 
    		if (findstring PH[NoNum].name "Left")!= undefined then PHLa[NoNum] = "L"
    		if (findstring PH[NoNum].name "Right")!= undefined then PHLa[NoNum] = "R"
    		)
    		
    ---------------------------------------------------------------
    -- transform PointHelper
    ---------------------------------------------------------------	
    		
    	if chk =="OFFSET" do
    	(
    		Xpos=tes[2] as float
    		Ypos=tes[3] as float
    		Zpos=tes[4] as float
    		Noof[NoNum]=[Xpos,Ypos,Zpos]
    		PaNo[noNum] = NoNum-1
    		)	
    	
    	if chk =="CHANNELS" do
    	(
    		NoCh[NoNum]=tes[2] as integer
    		
    		)			
    	if chk == "{" do 
    	(
    		Level = Level+1
    		NoDo02 = 0
    		DeNo01[NoNum+1] = 0
    		DeNo02[NoNum+1] = Nodo02
    		)
    	if chk == "}" do 
    	(
    		Level = Level-1
    		NoDo01 = Nodo01 +1
    		NoDo02 = Nodo02 +1
    		DeNo01[NoNum+1] = Nodo01	
    		DeNo02[NoNum+1] = Nodo02
    		)
    )
    
    
    for i = 1 to Nonum do 
    (
    	print ChTo
    	if DeNo01[i] != 0 do
    	(
    		De= DeNo01[i] +1 
    		PaNo[i] = i - De
    		if PHLa[i] =="F" do 
    		(
 			if (findstring PH[i].name "Left") != undefined then PaNo[i] = LhNo
 			if (findstring PH[i].name "Right") != undefined then PaNo[i] = RhNo
    			)
    		if (findstring PH[i].name "Collar") != undefined then PaNo[i] = ChTo
    		if (findstring PH[i].name "Neck") != undefined then PaNo[i] = ChTo
    		)
    			
    	)
    	
    for i = 1 to NoNum do
    (
    	LaSe =Layermanager.getLayerFromName "BVH_PointHelper"
    	LaSe.addnode PH[i]
    	if i > 1 then PH[i].parent = PH[PaNo[i]]	
    	)
    
    for i = 1 to NoNum do 
    (
    	PH[i].position = Noof[i]
    	if i > 1 do
    	(
    		PH[i].position = Noof[i] +PH[PaNo[i]].position
    		)
    	)
    
    for i = 1 to NoNum do
    (
 	format"%	%	%	%	%	%	%	%	%	%	%	%
" "Node:" i "Level:" NoLe[i] "Label" PHLa[i] "Name:" "Parent node:" PaNo[i] "Channels:" NoCh[i] PH[i].name 
    	)
    
    	---------------------------------------------------------------
    -- draw joint spheres
    ---------------------------------------------------------------	
    	
    	NewLa "BVH_DisplayGeo"
    		
    	for i = 1 to NoNum do 
    	(
    		na01=PH[i].name
    		na02=filterstring na01 "_"
    		NaSp = "BVSp_" + na02[2] 
    		sp=sphere radius:Srad name:NaSp
    		sp.position = PH[i].position
    		sp.wirecolor = PH[i].wirecolor
    		sp.parent = PH[i]
    		PH[i].size = Srad*4
    		LaSe =Layermanager.getLayerFromName "BVH_DisplayGeo"
    		LaSe.addnode sp
    		)
    		
    ---------------------------------------------------------------
    -- draw bone lines
    ---------------------------------------------------------------	
    		
    	for i = 2 to NoNum do 
    	(
    		st= PaNo[i]
    		BS = PH[st].position
    		BE = PH[i].position
    		
    		na01=PH[i].name
    		na02=filterstring na01 "_"
    		NaLi = "BVLi_" + na02[2] 
    		
    		Li = SplineShape pos:BS
    		addNewSpline Li
    		addKnot Li 1 #corner #line BS
    		addKnot Li 1 #corner #line BE
    		Li.wirecolor = PH[i].wirecolor
    		Li.thickness = LDi
    		Li.render_displayRenderMesh = true
    		Li.sides = 6
    		Li.parent=PH[st]
    		Li.name=NaLi
    		LaSe =Layermanager.getLayerFromName "BVH_DisplayGeo"
    		LaSe.addnode Li
    		)
    	
    	Ro= rotateXmatrix 90
    	PH[1].transform = PH[1].transform * Ro
    ---------------------------------------------------------------
    -- Read Transformation
    ---------------------------------------------------------------
    
    seek BVF 0
    skiptostring BVF "Frames:"
    Rlin = readline BVF
    tes = filterstring Rlin "	, " splitEmptyTokens:false
    FraCou = tes[1] as integer
    
    Rlin = readline BVF
    tes = filterstring Rlin "	, " splitEmptyTokens:false
    FraTim = tes[3] as float 
    
    FraSec = floor (1 / FraTim) as integer
    FraRan = FraCou/FraSec
    
    frameRate = FraSec
    animationRange = interval 1 FraCou
    slidertime = 1 
    
    for i = 1 to NoNum do 
    (
    	PHBaMa[i] = PH[i].transform
    	)
    Pos00= PH[1].position
    
    with animate on
    (
    
    for i = 1 to FraCou do 
    (
    	slidertime = i 
    	Rlin = readline BVF
    	DaLin = filterstring Rlin "	, " splitEmptyTokens:false
    	
    	Xpos =DaLin[1] as float
    	YPos =DaLin[2] as float
    	ZPos =DaLin[3] as float
    	
    	Pos01 = [Xpos,Ypos,Zpos] + Pos00
    	
    	ZRot=DaLin[4] as float
    	XRot=DaLin[5] as float
    	YRot=DaLin[6] as float
    	
    	ZRMa=rotateZMatrix ZRot
    	XRMa=rotateXMatrix XRot
    	YRMa=rotateYMatrix YRot
    	
    	RMa=YRMa*XRMa*ZRMa
    		
    	PH[1].transform =  RMa * PHBaMa[1]
    	PH[1].position = pos01 * PHBaMa[1]
    	
    	DaIn=7
    	
    	for j = 2 to NoNum do 
    	(
    		No = PaNo[j]
    		Ma0 = matrix3 [1,0,0] [0,1,0] [0,0,1] [0,0,0]
    		Ma0.row4 = Noof[j]
    		PH[j].transform = Ma0 * PH[No].transform
    				
    		if NoCh[j] != 0 do 
    		(
    			NoZRot = DaLin[DaIn] as float
    			NoXRot = DaLin[DaIn+1] as float
    			NoYRot = DaLin[DaIn+2] as float
    			
    			DaIn = DaIn+3
    			
    			ZRNoMa=rotateZMatrix NoZRot
    			XRNoMa=rotateXMatrix NoXRot
    			YRNoMa=rotateYMatrix NoYRot
    			
    			RNoMa=YRNoMa*XRNoMa*ZRNoMa
    			RoMa=PH[j].transform
    			PH[j].transform = RNoMa*RoMa
    			
    			
    			
    			)
    		)
    
    )
    slidertime = 1 
    LaSe =Layermanager.getLayerFromName "BVH_PointHelper"
    LaSe.ishidden = true
    )
    
    

That is great markus, thank you very much!

Has anyone gotten some good results using scripts in the mean time? I would like to hear if it’s working or not for everyone.

Thanks.

STINGRAY_AUT ,

Works almost…

Run script, point demo_3.BVH, I see creation of pointhelpers “Spheres” “joints” “lines” “joints”

Is Demo.BVH 137 frames? i start max with 300 frames, but after i run your script, just before it errors, I see that there is only 137 frames. guess script read BVH and resets frames?

Are we working on the same BVH? Demo_03.BVH “77.2 KB (79,109 bytes)” “Monday, December 17, 2007, 6:52:28 PM”

Error i get is ” MAXscript File in exception ” – Type error: Call needs Function or class, got undefined.

Seems like it almost finishes, then error???

please reply ASAP…

Thank for your time and effort

 PEN

Good stuff. I should be in a need for more mocap soon and will be updating my TRC import tool to handle it.

Are we working on the same BVH? Demo_03.BVH “77.2 KB (79,109 bytes)” “Monday, December 17, 2007, 6:52:28 PM”

i got a different bvh file – ExampleBVH_1.bvh, 339kb 30.10.2007– so if you could supply me your demo file i will take a look at it …

   Markus 
   
   [ http://www.sb-vs.com ]( http://www.sb-vs.com/) 
Page 2 / 2