In this episode of The TWIML AI Podcast, host Sam Charrington interviews Andres Ravinet, Sustainability Global Black Belt for Data and AI at Microsoft. They discuss the impact of human activities on the environment, strategies for addressing environmental issues, and the application of AI in conservation efforts. They also touch on Microsoft's commitment to sustainability and reducing its environmental impact, as well as the role of ESG in sustainability reporting and Microsoft Sustainability Manager's capabilities.
Outline:
- Introduction to Andres Ravinet and his role at Microsoft
- The impact of human activities on the environment
- Strategies for addressing environmental issues
- The lack of data to measure and manage progress
- The application of AI in addressing environmental issues
- The United Nations' 'Early Warning for All' initiative
- Project Guacamaya and its use of AI in conservation efforts
- Microsoft's commitment to sustainability and reducing its environmental impact
- The role of ESG in sustainability reporting
- Microsoft Sustainability Manager and its capabilities
In this episode of The TWIML AI Podcast, Sam Charrington interviews Fatih Porikli, Senior Director at Qualcomm AI Research, about Qualcomm's research at CVPR 2024. They discuss the Clockwork units paper, which explores a text-to-image generator diffusion model using an autoencoder architecture. Fatih explains how perturbations in generative models have a compositional impact but do not change refined textures, and how changes in higher resolution layers significantly impact while changes in middle layers have a smaller impact. They also discuss the stochastic probing approach, which successfully extracts object-specific information in video-language models, and the fitness assistant paper, which introduces a large-scale dataset called Fit Coach for real-time on-device video portrait relighting. Fatih also shares Qualcomm AI Research's initiative to release their internal models, including the Clockwork model, to the community. Additionally, they touch on the Math Search paper, which challenges models to understand and extract information from images to answer questions, and the Speculative Decoding for Multimodal Language Models paper, which aims to improve language models by using a speculative decoding approach. Fatih also explains the concept of constant stylized learning or distributed learning across multiple people, and the challenges of evaluating networks on large prompt datasets. They also discuss the importance of optical flow in improving image quality, compression, and motion information in video, and the challenges of annotating optical flow data. Finally, Fatih shares details about the demos showcased by Qualcomm AI Research, including a fusion model that can seamlessly switch between different lurers and degrees of imposition without waiting, and Mobile Lava, which enables users to take a picture of an object and ask questions about it. The demos also highlight applications in autonomous driving, computer vision, segmentation, portrait relighting, and generating avatars. Additionally, Qualcomm coordinated workshops at the conference, including one focused on efficient large vision models, which is becoming increasingly crucial for running on edge devices.
Outline:
- Introduction to Qualcomm AI Research and Gen AI
- Discussion on the Clockwork units paper
- Explanation of perturbations in generative models
- Evaluation of generated images using metrics
- Introduction to the stochastic probing approach
- Discussion on the fitness assistant paper
- Explanation of the Fit Coach dataset and model
- Introduction to the Math Search paper
- Discussion on improving multimodal models with local visual information
- Explanation of speculative decoding in multimodal models
- Discussion on hierarchical resolution trees in AI research
- Introduction to segmentation-free guidance for text-to-image diffusion models
- Discussion on negative prompting in image generation
- Explanation of the challenges of evaluating networks on large prompt datasets
- Discussion on optical flow and its challenges
- Introduction to stereo-aware compression models
- Discussion on low latency neural stereo streaming
- Explanation of the parallel hyper codec architecture
- Discussion on the fusion model and Mobile Lava demos
- Introduction to applications of omnidirectional computer vision