Philip Meier

🚨Paper Alert 🚨 ➡️Paper Title: Diffusion Self-Distillation for Zero-Shot Customized Image Generation 🌟Few pointers from the paper 🎯Text-to-image diffusion models produce impressive results but are frustrating tools for artists who desire fine-grained control. 🎯 For example, a common use case is to create images of a specific instance in novel contexts, i.e., "identity-preserving generation". 🎯This setting, along with many other tasks (e.g., relighting), is a natural fit for image+text-conditional generative models. However, there is insufficient high-quality paired data to train such a model directly. 🎯Authors of this paper proposed “Diffusion Self-Distillation”, a method for using a pre-trained text-to-image model to generate its own dataset for text-conditioned image-to-image tasks. 🎯They first leverage a text-to-image diffusion model's in-context generation ability to create grids of images and curate a large paired dataset with the help of a Visual-Language Model. 🎯They then fine-tuned the text-to-image model into a text+image-to-image model using the curated paired dataset. 🎯They demonstrated that Diffusion Self-Distillation outperforms existing zero-shot methods and is competitive with per-instance tuning techniques on a wide range of identity-preservation generation tasks, without requiring test-time optimization. 🏢Organization:  Stanford University 🧙Paper Authors:  Shengqu Cai, Eric Ryan Chan, Yunzhi Zhang, Leonidas Guibas, Jiajun Wu, Gordon Wetzstein 📝  Read the Full Paper here: https://lnkd.in/gFZdz8-S 🗂️ Project Page: https://lnkd.in/gK36aXxe 🎥 Be sure to watch the attached Demo Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

23

A flawless mega-meme mashup powered by AI Luma AI has rolled out start and end keyframe support for their Dream Machine AI model this week. This feature enhances narrative control in videos, letting you define the initial and final frames of your scene, while text instructions guide the journey in between. https://lnkd.in/gYHgmwYd Source : Blaine Brown (@blizaine) ____ P.S. For my upcoming book 'Analytics to AI,' I'm gathering insights from leaders across the industry. DM me if you'd like to share your perspective. #AIKnowledge #AIForBusiness #AILeadership #sushtend #AI #GenerativeAI #LeadersOfAI

5

Granular control over Gen Ai is a must.. object by object consistency, neural assets. More intuitive, fast 3d tools combined with this technology is the way forward imo. Being able to switch between generating things with prompts entirely, to creating important elements by hand with a layer of gen AI as finish. Here I'm using Dreams on my PS5 combined with Gen 3 by Runway. Imagine how incredible it would be to be able to create like this in real-time! #art #3d

371

55 Comments

Has anyone used Meta Quest 3 or other AR/VR/mixed reality headsets? I'm looking for something that could be useful as both a capture device (sensors) and for AR/VR/MR/Vision related projects. If you have recommendations, please leave a comment. #AR #VR #ComputerVision #mixedreality

2

2 Comments

New smol-vision tutorial dropped: QLoRA fine-tuning IDEFICS3-Llama 8B on VQAv2 🐶 Learn how to efficiently fine-tune the latest IDEFICS3-Llama on visual question answering in this notebook 🤝 Find link to the notebook and the resulting model in comments 💬

326

5 Comments

Adversarial Diffusion Distillation (ADD) is a groundbreaking method for ultra-fast, high-fidelity image synthesis! ADD combines the strengths of diffusion models and GANs, generating stunning images in just 1-4 steps. This new approach significantly outperforms existing few-step methods, achieving state-of-the-art quality with unmatched speed.   Paper: https://lnkd.in/dxEAxqu8

5

NVIDIA just dropped Llama 3.1-Nemotron-51B, a super smart language model that's fast and accurate! It's like a better, quicker version of Meta's Llama-3.1-70B but uses fewer resources. This means it runs smoother on a single GPU and is much cheaper! 💸 • 2.2x Faster than previous models • Super Accurate without losing quality • Lower Costs for running AI models • Fits on a single GPU for big workloads Key Benefits: • Better accuracy at a lower cost • Optimized for inference and deployment • Works smoothly on cloud and data centers For Tech teams needing speed and efficiency and companies looking to scale AI services this is a game changer. The table below lists all the benchmarks we evaluated – comparing our model and the reference model – Llama3.1-70B. The ‘Accuracy preserved’ is the ratio between our model’s score and that of the teacher. ---- Follow for more Daily dose of AI! #AI #Chatbots #TechNews #LLM

25

4 Comments

Yesterday Nvidia launched Fugatto - a state-of-the-art foundational model for speech. It's a single model, trained on text-to-speech, text-to-music, speech modification, denoising and more. But regardless of the model architecture, you know what's interesting? More than 50% of the paper is dedicated to talking about how they generated datasets for training the model. They use libraries like pedalboard (by Spotify) to generate synthetically alternated samples - variations in speed, timbre, pitch, loudness etc. - and the model then learns to do things like "Make this audio faster". The moat in building multi-modal models will not lie in the architecture. It lies in the data sourcing and data pipelines. Whoever owns the largest source of speech, video data can develop the best models. This comes as a surprise to everyone who keeps asking us "What model do you use at smallest.ai?" Our answer usually is "How does it matter?" It's the data that matters!

477

21 Comments

S01E01 Seeds of Resistance- Quantum Resonance https://lnkd.in/gD3EZdyT Quantum Resonance Podcast Show Breakdown Special Preview Episode Title: S01E01 Seeds of Resistance Episode Summary: In this thrilling inaugural episode, we dive headfirst into the world of Quantum Resonance as Dr. Malik Evans, our protagonist, experiences his first leap into the life of David "Davey" Richardson. The leap is sudden and disorienting, throwing Malik into a high-stakes corporate environment in 1981, where Davey's company faces imminent layoffs due to Reagan's Economic Recovery Tax Act. Malik must quickly adapt to Davey's life, navigating tense interactions with his colleagues, Tom and Sarah, as he uncovers a web of corruption and conspiracy. Key Themes: Time Travel and Quantum Mechanics: Explore how Malik's leap is not just about correcting wrongs but understanding the deeper, hidden order within chaotic situations. Corporate Greed vs. Economic Justice: The backdrop of Reagan's economic policies provides a rich setting for examining the impact of wealth inequality and corporate corruption. Identity and Deception: Malik's struggle to embody Davey while maintaining his own sense of self highlights the tension between outward appearances and internal truth. Characters Introduced: Dr. Malik Evans: Our protagonist, a quantum physicist who leaps into the lives of others, tasked with uncovering the hidden perfection in every scenario. Tom: A suspicious colleague of Davey's, revealed to have connections to a radical group with potentially dangerous intentions. Sarah: A journalist with her own agenda, playing both sides in a dangerous game of espionage. Admiral Julius "J.T." Thompson: Malik's disembodied guide, providing crucial information through The Echo, a quantum device that only Malik can interact with. Plot Highlights: Malik, disoriented after his leap, quickly realizes he has assumed the life of Davey Richardson, an accountant in a corporation on the brink of collapse. Tom and Sarah confront Malik, pushing him to act decisively in a situation fraught with danger and suspicion. Malik learns from J.T. that Tom is linked to a radical group, and Sarah is more deeply involved than she appears, escalating the tension. As Malik tries to maintain his cover, he receives a steady stream of information from J.T., challenging his ability to make the right moves without revealing his true identity. https://lnkd.in/gD3EZdyT #QuantumResonancePodcast #TimeTravel #QuantumMechanics #CorporateGreed #EconomicJustice #IdentityStruggles #PodcastLaunch #ThrillerPodcast #QuantumLeap #PodcastCommunity #Storytelling #SciFiPodcast #InnovationInStorytelling #QuantumPhysics #NewPodcastEpisode #LeadershipInCrisis #CorporateConspiracy #PodcastRecommendations #SciFiThriller #AntonioTSmithJr

2

1 Comment

Innovations in this area are on an exponential path

2

1 Comment

Metas new model Chameleon blew my mind 🤯 It's multi-modal, so text and images, but it does so using the same encoder! Unlike image gen models, it uses tokens not diffusion! To do this, it has to convert the text and images into a single sequence of tokens when processing. Then, a single transformer processes these mixed-modal tokens, eliminating the need for a different encoder per modality. If this works well, it might make for more scalable and efficient multi-modal models! I'll put a link to run it in the comments 🤙 Note: they made it "safety-aligned" and ripped out the ability to generate images, however, that code is still in there. If you are able to unlock it, I'll give you $1000

125

3 Comments

I just came across a fascinating paper that proposes a novel approach to reconstructing 3D scenes from a single view using a two-bounce lidar. I know, I know… 🤓 Still, check this out! One of the key challenges in 3D scene reconstruction is the need for multiple views of the scene. Traditional methods rely on capturing images or videos from different angles to build a complete 3D model. However, this can be time-consuming and impractical in many situations. The new approach proposed in this paper uses a single-photon avalanche diode (SPAD) lidar to capture time-of-flight data. This data can then be used to reconstruct the 3D scene using a neural radiance field (NeRF). NeRFs are a powerful tool for 3D scene reconstruction and are often underutilized. In other words, we’re looking at shadows to reconstruct entire scenes. You know the whole allegory of Plato’s cave? That’s the inspiration for it, hence the whole PlatoNeRF idea. Here’s the paper itself: https://lnkd.in/eVKyCYGD #3DReconstruction #NeRF #LiDAR #ComputerVision

7

#VD3D: Taming Large #VideoDiffusion Transformers for #3DCameraControl. I have shared approaches from others before, but this is a straightforward "pick the camera movement you like from anything, and then have it transferred to any video diffusion prompt you have, for the control you seek."   Some approaches demonstrate the ability to generate videos with seemingly controllable camera poses — these leverage pre-trained #UNet-based diffusion models that disentangle spatial and temporal generation.   Presently, no setup enables camera control for novel, transformer-based video diffusion models that process #spatialtemporal information together.   Nerd Details: the authors of this paper's proposal are 3D camera control using a #ControlNet-like conditioning mechanism that unifies spatiotemporal #camera embeddings, based on Plücker coordinates (this is a means to assign six homogenous coordinates, to each X, Y, Z line in projective 3D space). Without getting more technical, suffice to say that these create a quadratic constraint that keeps them constant in space-time.   For more details, check out the project page: https://lnkd.in/gGWfAzi5   #research paper: https://lnkd.in/gqDje2X6

2