#34 · Primary category: 3D Generation & Asset Creation

instruct-nerf2nerf

3d computer-vision diffusion-models machine-learning nerf pytorch

Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions (ICCV 2023)

Project last updated:02/12/24

GitHub Stars

853

Forks

80

Contributors

4

License

MIT

Why we included this project

Every now and then you build a 3D scene from photos and then want to change it without starting over. Instruct-NeRF2NeRF is the official implementation of the ICCV 2023 paper of the same name, and it takes a practical path to that problem: it builds on Nerfstudio, uses the InstructPix2Pix diffusion model to touch up images a trained NeRF was fit to, then retrains so the edits hold from every viewing angle. A short text instruction is all it takes to change how the scene looks. This is research code, not a finished product. You train a regular NeRF first, then run the in2n trainer with a prompt and guidance settings; expect to sort out PyTorch, tiny-cuda-nn, and a few docs before it clicks. For teams exploring product visualization or set design, it is a strong base to adapt, but plan for setup time.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category