#ByteDance showcases #realtime fully #genaivideo, with #avatars of all types. What's freakish is that these execute interactive *non-verbal* cues, as they listen and respond. That is the kind of artful subtlety that can *manipulate people* with ever more convincing #genai.
I've presented a company that does this via an API with super low latency using a kind of #deepfake, but this version by ByteDnce (which owns #TikTok), can use content that is fully synthetic, or an alteration of a real person.
This is happening simultaneously in #volumetric applications, so this can exist in #videogames, #vfx, #animation, and #xr.
Abstract:
INFP: Audio-Driven Interactive Head Generation in Dyadic Conversations
(Dyadic = social interactions between two persons)
We present INFP, an audio-driven interactive head generation framework for dyadic conversations. Given the dual-track audio in dyadic conversations and a single portrait image of arbitrary agent, our framework can dynamically synthesize verbal, non-verbal and interactive agent videos with lifelike facial expressions and rhythmic head pose movements.
Additionally, our framework is lightweight yet powerful, making it practical in instant communication scenarios such as the video conferencing. INFP denotes our method is Interactive, Natural, Flash and Person-generic.
I must add that without the availability of the code, it's difficult to emulate. Given what I have seen from other companies, from cloud LCM models for 2D, 3D, lip-sync, and the plethora of gen ai video updates from the likes of #OpenAI and #Sora to #Google's #Veo2 ("veo" means "I see" in Spanish), this seems like a naturally progressive jump.
Project Page: https://lnkd.in/gjZdpC5f
***
"The Party told you to reject the evidence of your eyes and ears. It was their final, most essential command." - George Orwell, "1984"
What do you think of the last time we saw each other? What do you remember feeling? So the last time was in Santa Cruz? Yeah, it was like mid-october. Yeah. I remember being happy that I could see you, but I really felt like I was. I kind of like didn't belong. I'm thinking like, I have to schedule things to do, like find things. And you're, you're OK with just like hanging around, like getting done what needs to be done in time. Yeah. And I like, and I am. I think what we were both doing was like judging the other based on what you think is best for yourself or what you think is important for yourself. And that's where we got our wires crossed. Like, I'm not mentally ready for this. Ohh. Nobody's ever really ready for it. I feel like some people are ready for it. More ready for more, more, more ready, yeah. But what about yours? But I feel really engaged in everything you're saying and intrigued by what you're saying. So for me, yeah, I don't know, I feel a little connection. Yeah. You know, it's like it's a, it's a big mole right there. You know, it doesn't have to be physical, but sometimes it was or sometimes it was distance or whatever, right. And we were pretty far. We were far just worked out, you know, timing worked out. I don't know if that means anything. But that means something, no? It's understandable when you're asking for hugs and I just don't want to be held. I want to be held at that time. Well. I do. I know. Yeah. I mean, a lot of the time my mind goes to like, worst case scenario things and you are able to calm me down and bring me. You're very positive. Yeah. And you can kind of bring me back to reality. ****. Why did this go so well? I'm scared. You were terrified that it went well. Yeah. Cuz it was like it was weird that we just met. We spent all this time together, like so much time. But it was so comfortable. It was kind of strange how comfortable it were around each other. It was really weird and I never experienced that with anyone. I'm like, how do you define life? Yeah, yeah. What level of life are we talking here? Right. I lied to my friend this morning when he asked me what am I up to today. I said, well, I'm just going to do a couple. Yeah. I mean, a lot of the time my mind goes to like worst case scenario things and you are able to calm me down and bring me. You're very positive. Yeah. And you can kind of bring me back to reality or whatever my reality is, yeah. When I'm working on something creative, I do it for myself. That's my way to have a conversation with myself. Yes, you know, and to kind of like let my feelings out. I think there are like one or two people in the world that like, wrote that. Read what I wrote here. Really, really careful about love singing too, but. No. To my love. But don't let anyone. Il est fabriqu�� avec un mat��riau de haute qualit�� qui garantit confort et durabilit��, le rendant adapt�� �� toutes les occasions. Have There are three very, very capable lawyers from the Cobb County District Attorney's Office sitting there at council table. Linda Donna Koski has lit the court.
They’re either developing this tech to build virtual influencers or to replace humans stuck in endless corporate zoom meetings. Joke aside, we’re getting to the point where we cross the uncanny valley here. Once beyond it, we won’t be able to believe anything we see on social media.
GenAI Advisor to Creatives 🤖 | Minimax, Nim Creative Partner 🌈 | Thought Leader & Speaker 🎤 | Creator Economy Veteran 🎥 | Ex-YouTube
14hThey’re either developing this tech to build virtual influencers or to replace humans stuck in endless corporate zoom meetings. Joke aside, we’re getting to the point where we cross the uncanny valley here. Once beyond it, we won’t be able to believe anything we see on social media.