XDXL 0.9 Availability:
XDXL 0.9 is currently available for use on ClipDrop and will soon be released on the API, Green Studio 2, and research weights. Research weights are being distributed to a select group of 2,000 sign-ups and are not yet finalized.

XDXL 0.9 Features:
XDXL 0.9 offers dynamic aspect ratioing, excellent inpainting and uncropping capabilities, and improved performance on image-to-image generation. The model is designed to work consistently across different platforms, including Dream Suite and the API.

Future Plans:
A full 1.0 release of XDXL is targeted for mid-July, with significant improvements over the current version. The 1.0 release will be licensed under a Creative ML license, allowing users to extend and use it as they wish. Future releases will focus on continuous improvement and integration with various tools and platforms.

Open-Source Tools and Community Involvement:
Stable Diffusion Studio, a collection of open-source tools and interfaces, will be available to enhance the creative process and provide users with more control over their creations. The team is committed to fostering community involvement and collaboration to drive future developments and improvements.

SDXL vs. GPT-4:
SDXL, like GPT-4, is a large model that requires subsequent refinement.

Data Optimization:
The team is working on optimizing datasets to improve performance and accessibility. Funding initiatives like Datacomp and Washington to develop better datasets.

Making SDXL Accessible:
The goal is to make SDXL accessible on various devices, including MacBooks. Refinements include improving data, architectures, and accessibility.

Evolution of Benchmarks:
Current benchmarks for SDXL are subject to change as the model continues to evolve. The architecture will change before the 1.0 release. Improvements and optimizations will emerge over time.

SD 1.5 vs. SDXL:
SD 1.5 is still useful for certain tasks and can be used alongside SDXL. SDXL is for users who desire top-of-the-line performance and are willing to invest in more powerful hardware.

Hardware Requirements:
Confirmed that 8 gigabytes of VRAM is sufficient for image generation. Full 1024×1024 training at rank eight is possible on an RTX 2070 with 8 gigabytes of VRAM. Affordable options like the RTX 2070 and RTX 3060 with 8 gigabytes of VRAM can run the model.

Optimization Efforts:
The team is working to optimize the model’s VRAM usage. Initial versions of Dreambooth required 40 gigabytes of VRAM, but it has been reduced significantly.

State-of-the-Art Model:
SDXL is likely one of the state-of-the-art models in the world. Users can continue using SD 1.5 for specific tasks.

Main Improvements in SDXL:
Improved prompt compression resulting in better adherence to prompts and superior performance compared to other models. Enhanced natural language interaction, enabling users to generate content using conversational language. Minimum bar of capability significantly higher, allowing users to generate impressive results with minimal effort.

Stable LLM and Stable Diffusion Integration:
Plans to integrate Stable LLM and Stable Diffusion, similar to Bing Chat and Dolly, are in the works but currently on hold due to other project commitments. A language model UI built by the team is ready, and the new release of StableRM is being prepared to accompany it. High-performing base models, including Lama replications up to 20 billion parameters, are in development. Work is underway to create an instruct model, with front-end code ready to be open-sourced.

Energy Consumption and Optimization:
Considerations for reducing energy consumption are being made, aiming to optimize the model for efficiency. Efforts are being made to develop the model for use on devices with limited resources, such as MacBooks. The goal is to achieve a balance between energy consumption and creative potential.

Distillation and Release Plans:
Distillation of the model is challenging due to the loss of fine-tuning capabilities. The team is aiming for a mid-July release but may release a 0.95 version if necessary. Continuous improvement and periodic benchmark releases are planned to ensure a stable and high-quality model.

Grant Program for Fine-Tuners:
A grant program is being considered to support fine-tuners and experimentation with the model. The goal is to encourage exploration and development without significant resource investment before the model stabilizes. This program aims to foster a collaborative community and drive innovation.

Model Training and Data:
SDXL 0.9 is a combination of various data sets, including proprietary and other sources. The training process is very tunable, allowing the model to learn quickly and adapt to different use cases.

NSFW Filtering and Exclusion:
SDXL 0.9 is capable of generating NSFW content due to its diverse training data. However, the model can be adjusted to reduce or eliminate NSFW content as needed.

StyleDrop and Future Model Evolution:
StyleDrop and other recent advances are expected to diminish the importance of tuned models like Dreamers and LORAs. The focus is shifting towards real-time inferencing and retrieval-based models.

Inference Speed and Model Size:
SDXL 0.9 is faster for high-resolution inference despite being a larger model. This is achieved by removing exponentially growing parts of the model, resulting in overall faster performance.

Memory Consumption:
SDXL 0.9 requires more memory compared to previous models, but the increase is manageable. The community is expected to find ways to optimize the model for various hardware, including lower-end GPUs.

Instructing Smaller Models:
SDXL 0.9 can be used to instruct smaller models, following a similar approach to language model training. This will allow for more efficient and diverse model usage.

Model Development Process:
SDXL 0.9 is described as a “Frankenstein amalgam” due to its combination of different training data and techniques. The model has been in development since February 2023, undergoing various iterations and adjustments.

Model Optimization and Performance Improvement:
Future optimizations can lead to significant performance improvements and reduced computational complexity in various model types, including portraits and architecture.

Video Consistency and Temporal Coherence:
Video models with temporal consistency require multiple model modes interacting with each other. Current methods of interacting with models are limited and can be improved.

Chaining Models for Enhanced Control:
Combining image, temporal coherence, and 3D models with the ability to exchange between them can lead to greater control and flexibility. Control nets, segmentation models, and other techniques can be applied to achieve dynamic adjustments and fine-tuning.

Explosion of Model Types:
An array of new model types is expected to emerge, requiring excellent guides, guidelines, and patterns to help users make the most of these models.

Consistent Character Sheets:
The new architecture shows promising results in generating consistent character and face sheets for production purposes.

Workflow for Building Large Models:
Building large models like SDXL involves continuous training and experimentation. Researchers try multiple approaches and only move forward with models that show significant advancements. Extensive research and ablations are conducted to optimize and refine the models.

Process of Iterative AI Model Development:
The Apply team received base models from the research team led by Robin. To refine the models, the teams conduct various tests to identify the best-performing ones. Initial models are often imperfect and may produce undesirable results. The process involves branching out from the base models and gradually improving them.

Considerations for Model Improvement:
Different models may excel in specific domains, such as photorealism or anime-style art. The architecture of the model plays a role in its strengths and weaknesses. Models are like precocious toddlers that require careful nurturing and training.

Community Involvement and Exploration:
The community has used past base models to create various extensions and modifications. The Apply team investigates methods to enhance the base model for community use. Popular techniques like Loras and Dreambooths are tested on different architectures to identify the most effective combinations.

Continuous Training and Progress:
GPUs are utilized to ensure continuous training and progress in model development. The team monitors loss graphs and sample outputs to assess model performance. Ongoing exploration and experimentation lead to new insights and improvements.

Updates on SDXL and Commercial Terms:
A new Terms of Service is in the works for the main release, available on AWS with Bedrock service. Fully licensed versions, an opt-out version, and synthetic datasets are coming. Stability.AI aims for continuous improvement based on user feedback. The goal is to ensure enterprise readiness, a supporting ecosystem, and user-friendly tools before aggressive monetization. Partnerships with companies like Amazon aim to ensure wide availability for commercial and personal use.

Focus after SDXL Launch:
All of the above! Video models, specialized models, photorealistic and digital art are all in the works. Dedicated teams for video models and future Stability projects. Collaborations with chip manufacturers to make models accessible on various hardware.

Team Expansion and Goals:
Stability.AI’s team has grown to 86 people, with more hires planned. Democratization of AI technology is a key goal, enabling usage on commonly available hardware. The team is excited about SDXL’s output quality but sees room for improvement. Streamlining and reducing bulkiness in future models are expected. New architectures, like SD3, are being explored.

Comparison with Mid-journey:
Stability.AI and Mid-journey have different objectives. Stability.AI focuses on open-source research and building foundational blocks for others to build on. Mid-journey aims to create a real-time 3D video engine for video games. Both organizations are growing and pushing the boundaries of AI technology.

Audience Participation and Community Engagement:
Emily expressed gratitude for the audience’s involvement and questions, which contribute to the community’s journey and growth.

Future Events and Collaborations:
Emily announced that there will be more similar events in the future, encouraging attendees to participate and contribute their ideas and experiences.

Community Support and Involvement:
The speaker acknowledged the importance of community involvement and collaboration in achieving shared goals and fostering a positive experience.

Appreciation for Attendees’ Time and Engagement:
Emily thanked the audience for dedicating their time to attend the event, listen attentively, and actively engage in the discussions.

Speaker’s Excitement and Motivation:
Emily conveyed her enthusiasm and motivation in organizing these events, emphasizing the potential for creating a positive and impactful experience for the community.