大家好。几周前我分享了我的参考资产编译器(RAC),这是一个基于 AI 模型的 3D 资产构建管道,旨在提供最高保真度的输出。为此演示,我在大约 13 小时内创建了 121 个资产。这包括所有资产的创建、贴图、反馈、重新渲染等。演示中的大多数物体是一次性完成的,但有几个需要重新贴图,或网格出现了问题。

我继续了这项工作,并想分享整体的进度报告。此外,我开发了一个前端 UI,能够与 RAC 协同查看输出,名为 Framewright。

以下是仓库。两者均为开源:

RAC -- https://github.com/raydeStar/reference-asset-compiler

Framewright -- https://github.com/raydeStar/framewright

这里简要概述,如有疑问请提问 --

RAC

Image-to-3d pipeline. Supports 3d objects, texturing, rigging, animation, and tested specifically on the UE5 mannequin skeleton. Work will probably have to be done to get it into Unity or Godot, but I tried to keep that minimal.

RAC Future Plans

Right now I am working on facial expressions and talking. I am about 90% of the way there, I think, but there are still some issues I am working through.

Framewright

I built this originally as a pipeline to create videos using Minimax H3. I wanted to have a storyboard, a pipeline, and easily available hooks. Afterwards I added the full setup for development -- 3d models, images, Audio (speaking), and music (YUE2 is plugged in right now).

The idea behind this one, is to link mostly directly to ComfyUI and set up the workflows without you having to touch it. When a new update comes across, when new workflows come, you can just ask Codex or Claude to pull it, set up the models, troubleshoot, optimize, etc. Basically, it's a layer of abstraction above your existing framework, to get out work quickly. Also, it supports linking in with Codex oauth, and you can have it create images for you.

It also supports concept images and a wiki of characters you've made for consistency. This is all to say -- it's a planning board for video, image, 3d model, music, and voice production.

Framewright Future Plans

Someone online decompiled adobe projects and open-sourced them. I am planning on onboarding a few pieces from there, so it can work in tandem with what already exists. Mostly, I am interested in a more robust photoshop for images, giving our users more leverage over shaping a 3d scene, and overall borrow production-grade AAA concepts to make it feel like a good product.

Workflow

Workflow for me was simple. First, I asked Claude to build me a dungeon as a demo. It took items that already existed, and placed them on the screen. Then, I asked it to create a requisition request, and it did.

I then passed it to Astra. Astra facilitated all the concept images, and models autonomously. I was the approver. 90% of the models passed, but the more complex ones needed to be sent back. Also, for things like the gems and metallic objects, I had it do a materials pass, to show the right reflectivity, opacity, etc.

When everything was made, I sent it to UE5. The video you see is after some spitballing back and forth with placement -- it was not a 1-shot process, but it was pretty close.

Thanks for your time guys! I'd like to make this more useful for everybody -- very open to feedback!