Discussion

• @awinkler_ Alexander Winkler on x

  This paper shows some of our research at Meta Reality Labs in reconstructing a user's pose from only the sensors of the Quest headset using Reinforcement Learning. authors: with Jungdam Won and Yuting Ye paper: https://arxiv.org/... video: https://www.youtube.com/... 1/4👇 https:/…

• @drgbuckingham Gavin Buckingham on x

  this is pretty astonishing https://twitter.com/...

• @thegarpinvestor Ethan Tucker on x

  Watch this and tell me that $META has no chance of success. VR/AR is a very long term bet, but one that could transform the way we live. Zuck is pushing in his chips and I'm not going to bet against him. https://twitter.com/...

• @diegobez Diego Bez on x

  My bet is that full body tracking will be another game changer for VR, when it reaches “enough” accuracy. The limit tonfull body trscking seems to be the cameras position ( on the headset). Will the new Quest Pro controller's cameras be used for good full body trscking in Quest2?…

• @chrismessina Chris Messina on x

  LEGS IN THE METAVERSE!!! https://twitter.com/...

• @hongzimao @hongzimao on x

  I just read the paper. Like how simple it is. You basically just need to predict the location 5 points in space (hip, left/right knee, foot). It can be done based on a time series of existing upper body tracking (head, hand, height) and physics. https://twitter.com/...

• @hongzimao @hongzimao on x

  Interesting use of Reinforcement Learning too (though the key here is to use RL action to control physical simulation, not really the RL exploration-feedback loop). I've been waiting to unleash the use cases of RL.

• @simesgreen Simon Green on x

  This is a very cool system - it generates a full body pose from just the position+orientation of the headset and hands, using machine learning and a physics model. It doesn't even know where the feet are! https://twitter.com/...

• @mhdempsey Michael Dempsey on x

  It feels inevitable to me that: 1) The default input device for AR/VR is mostly the body in the long-term 2) ML will continue to render other forms of sensing obsolete 3) The camera is the best sensor in the world https://twitter.com/...

• @bradneuberg Brad Neuberg on x

  This is really cool work. The impact of full body tracking can be really nice in social VR apps like VR Chat, and it really increases immersion and makes it easier do things like dance at VR music/DJ fests (which exist and are quite fun) https://twitter.com/...

• @glassbottommeg Megan Fox on x

  The key here is that this is meta researches. Their priorities aren't yours. They want to make Meta avs with legs that pass visual fidelity bars. They don't give a good diddly how it feels to inhabit them, their goal is to appease investors and give marketers pretty footage.

• @nathievr @nathievr on x

  Wow! So the Quest 2 can actually pull off full body tracking without the need of any extra trackers. Very impressive and exciting news for social apps like VRChat. https://twitter.com/...

• @fbbagholder @fbbagholder on x

  I'm understand by the day why $META is spending so much on their ai supercluster. I understand it's use cases for the metaverse, but will it show results for ad attribution?

• @alexqgb Alex Bowles on x

  The gap between what they're doing and what they're selling remains astonishingly wide https://twitter.com/...

• @fbbagholder @fbbagholder on x

  Full body tracking with only Headset and Controllers as input. That's with Quest 2 controllers, not even Cambra controllers (with cameras of their own) Quest has a history of introducing software that makes the product much better. Impressive stuff! https://twitter.com/...

• @json_dirs @json_dirs on x

  I really want to see the network estimate hand gestures during conversation, or do they just sort of hang there while you bob your head around like a fish

• @joewintergreen Joe Wintergreen on x

  this video leans so hard on your inability to look at both sides of it at once https://twitter.com/...

• @dingemansemark Mark Dingemanse on x

  @json_dirs Also raises questions about social accountability. Can I be said to have made a certain gesture or posture when it actually was my avatar generated based on merely head movement? Am I the animator/author/principal of said avatar?

• @tom_forsyth Tom Forsyth on x

  @glassbottommeg I do remember some early research that had “AI” eyes - when you spoke, they looked at you, and when the user was speaking, they looked at something indeterminate. And people really liked those way more than actual eye-tracked eyes. Because real eyes JUDGE YOU.

• @tbwrightartist @tbwrightartist on x

  I'm fully convinced that the “democratization” of mocap will be one of the biggest boons for the game industry. When a single developer can do a full motion set in their living room with an off-the-shelf product ohhhh that's where the magic is https://twitter.com/...

• @skarredghost @skarredghost on x

  Not perfect, but still incredibly cool Meta AI Research: Quest 2 Body Tracking Estimation Without Trackers https://uploadvr.com/... via @uploadvr #VirtualReality #VR

• @nathievr @nathievr on x

  It is not perfect yet but promising. As of right now there are some challenges such as latency and more. You can read the full paper named QuestSim here: https://arxiv.org/...

• @manan @manan on x

  fuck me. Quest going to teach me how to stand straight or make me bow down to Zuckerberg's portrait?? https://twitter.com/...

• @json_dirs @json_dirs on x

  in the metaverse we'll all be generic sort of drunk guys whose only body language is a complex Morse code of shaking our foreheads around https://twitter.com/...

• @amar_5261 Amarnath Murugan on x

  @awinkler_ This is amazing! We tried to tackle the same problem but with just a phone that uses ARcore. However, our approach used classical procedural animation & blend trees instead of ML/DL https://twitter.com/...

• @glassbottommeg Megan Fox on x

  Um This isn't full body, this looks about as good as various IK solutions, which are “fine” but don't remotely feel like Fullbody tracking. If your elbow or pelvis or knee is wrong, it doesn't matter if it's a little wrong or a lot wrong, “imprecise” all feels about the same. htt…

• @glassbottommeg Megan Fox on x

  Which I'm sure they know, and they don't care, because this is tech they're going to use to make VR avs look more realistic to other people, because it makes their apps look better and easier to sell. That's, fine, but that isn't why people use Fullbody in VRChat or similar.

• @dennycloudhead @dennycloudhead on x

  This is super promising & part of what I was yakking about wrt industry strides towards frictionless tracking solutions. This will very likely replace our best IK attempts & open a new world of interaction, immersion, socialization potential. https://twitter.com/...

• @tom_forsyth Tom Forsyth on x

  @glassbottommeg As long as you can make “virtual meeting rooms” feel realistic, you get the VC cash. I'd say that's a stupidly low bar, but if you've ever physically met with any VCs, getting them to impersonate humans is... uh...

• @rschu René Schulte on x

  Full body pose only from head and hand tracking! 🤯 We are getting close to full body realistic avatar movements with this #AI approach by Meta's Reality Labs leveraging Reinforcement Learning. They also have a version that only uses the head pose to inference the body pose. 👏 htt…