Android tracking

FlyingBanana78
FlyingBanana78 Posts: 502 Enthusiast

I just happened to stumble across this on youtube and figured it would be a letdown as apple seems to have the lockdown on smartphone and tablet tracking. But I am rather surprised at the quality of the results and I am not using a high end android phone and it seemed to track facial movement and camera track rather nicely. That said the CamtrackAR is still by far the better option if you have an apple device but seeing this guys work on the android side of things it looks promising. And he is still tweaking it to be better and more user friendly. He has a link in the youtube description for the blender addon to be able to import your tracking data.



PS Not trying to take away from FXHome with CamtrackAR just wanted to share some additional options to the android users out there. And maybe you all can get with this guy to possibly figure out a Hitfilm addition.

Comments

  • Stargazer54
    Stargazer54 Posts: 3,800 Ambassador
  • FlyingBanana78
    FlyingBanana78 Posts: 502 Enthusiast

    @Stargazer54 figured I would share it for the android users out there that want to do some phone tracking stuff. And I guess he is also doing a mocap through webcam that tracks hands/fingers head and torso.

  • DreamArchitect
    DreamArchitect Posts: 604 Enthusiast

    @FlyingBanana78 I just found this myself about a week ago. I tried using this to get data into blender and it works great. The app itself is a little clunky in places but given one guy made it it's pretty good and shows that it can be done on android. Over to you fxhome.

    Anyway I tried the following workflow. Blenderartrack tracked an outdoor scene.

    Import to blender with the supplied python addon. Works well and seems to have tracked in blender accurately on the simple clip I created.

    Exported camera using the blender to hitfilm addon.

    Imported the comp shot to hitfilm and also

    Imported the mov file shot with blenderartrack. This required the extraction of files from the zip file created by blenderartrack. Added footage to the composite shot.

    Added a point and made it 3d.

    This is where the problems show up. There is not the same lock between footage and camera as there is in blender.

    I have tried the frame rates of the blender camera compared to the footage. It seemed like blndercam was 24 fps and movie was 30 fps. So tried changing blender cam to 30fps. No luck but in blender it still works fine.

    Tried changing the height of the hitfilm camera and that helped a bit but still not right.

    My thoughts are this should work. Anyone know what I've missed or made this work?

  • FlyingBanana78
    FlyingBanana78 Posts: 502 Enthusiast

    @DreamArchitect yeah I haven't tried exporting into hitfilm with it. Been still kind of under the weather, but did see he is still working on all his addons. He also has a motion capture addon that does face/hands/arms/legs. So I'm sure there will be more to come for it.

    Also found a photogramatry android app that works rather well too. The only downside is you have to upload the result to sketchfab then you can download it once you upload.

    That is a scan I did with it and there is no cleanup done to the scan, just scanned and uploaded to sketchfab. It's called 3d Live Scanner.

  • DreamArchitect
    DreamArchitect Posts: 604 Enthusiast

    @FlyingBanana78 that looks great.

    What android device did you use?

  • FlyingBanana78
    FlyingBanana78 Posts: 502 Enthusiast

    @DreamArchitect I have just mid level samsung a51 not high end phone by any means.

  • DreamArchitect
    DreamArchitect Posts: 604 Enthusiast

    @FlyingBanana78 thanks for the info. I have an s20 ultra so will give the free version a go first

  • FlyingBanana78
    FlyingBanana78 Posts: 502 Enthusiast

    @DreamArchitect Yeah I am using the free as well.

  • bitcohen
    bitcohen Posts: 12 Just Starting Out*
    edited April 20

    Great features, easiest workflow. Thank you so much for sharing this. I have been searching for automatic AR motion tracking for so long.

    Arkit seems easier to work with and is helped by the truedepth camera or whatever it is on iOS devices. I'm sure these things will come to android at some point but iOS got a head start on AR for sure.

  • DreamArchitect
    DreamArchitect Posts: 604 Enthusiast

    @bitcohen some android devices have the depth camera too. This app is proof it can be done on both platforms. I'm gonna have a look at the unity code when I get a spare bit of time and see how this app works. It can be improved upon for sure but for one guy thats really good work. Especially with the mocap as well.

    I don't really work with apple devices so can't speak for arkit vs arcore and ease of working with. My experience is that android devices seem to detect surfaces quicker and more accurately but I have only tried one apple device with camtrackar and it's not top of the line so might just be down to that. My android devices tend to be flagship ones so in general will perform better.