Skip to main content
Published in Product Insights

Repo Roundup June 23rd

Repo Roundup June 23rd
Table of contents
  1. VGGT
  2. What is it?
  3. What does VGGT predict?
  4. Which award did the VGGT paper win?
  5. How many images does VGGT take, and how fast is it?
  6. Why is this important?
  7. Practical Uses
  8. Can you use VGGT commercially?
  9. Demo
  10. How do you run VGGT on your own photos?
  11. What files can VGGT export?
  12. Can you train or fine-tune VGGT yourself?

VGGT#

https://github.com/facebookresearch/vggt

While drafting my Repo Roundup post this week, I was intrigued by the VGGT project that came out of Facebook Research. Rather than review a few projects, I wanted to spend some time diving deeper into this one  to understand it better. It is very much still in research project mode, but as a relative layperson in this area, I think I can see where it's headed from a practical use perspective.

What is it?#

This project is demo code from an award-winning paper titled "VGGT: Visual Geometry Grounded Transformer." It essentially takes one or many images (or video) of a subject and reconstructs it as a 3D scene. Behind the scenes, it uses a neural network to predict cameras, point maps, depth maps, etc., as opposed to more simplistic solutions that more or less attempt to stitch scenes together or actual 3d scanning hardware. It is capable of doing these calculations in less than a second, so very close to real-time.

What does VGGT predict?#

The README says the model directly infers extrinsic and intrinsic camera parameters, point maps, depth maps, and 3D point tracks. It does this from "one, a few, or hundreds of its views." The post says VGGT "takes one or many images (or video) of a subject and reconstructs it as a 3D scene."

The README's Detailed Usage example shows each output in turn.

OutputWhat the README says
Camera parametersThe code comment reads "Extrinsic and intrinsic matrices, following OpenCV convention (camera from world)".
Depth mapsThe example returns depth_map and depth_conf together.
Point mapsThe example returns point_map and point_conf together.
3D points from depth maps and camerasThe comment says this "usually leads to more accurate 3D points than point map branch".
3D point tracksThe comment reads "choose your own points to track, with shape (N, 2) for one scene".

The paper adds three details about the inputs and outputs.

  • "The order of the images in the input sequence is arbitrary, except that the first image is chosen as the reference frame."
  • The 3D points "are defined in the coordinate system of the first camera."
  • The paper assumes the camera's "principal point is at the image center."

The README also says you can "optionally choose which attributes (branches) to predict."

Which award did the VGGT paper win?#

The post says VGGT is "demo code from an award-winning paper," and links the word "award-winning" to CVPR. CVPR's awards page, announced in Nashville on 13 June 2025, lists the paper as "CVPR 2025 Best Paper: VGGT: Visual Geometry Grounded Transformer."

DetailWhat the source says
AuthorsCVPR lists Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht and David Novotny.
AffiliationsThe README credits the "Visual Geometry Group, University of Oxford" and "Meta AI".
First postedarXiv shows version 1 submitted on Fri, 14 Mar 2025.
README noteIts June 13, 2025 update says "Honored to receive the Best Paper Award at CVPR 2025!"

CVPR's page also describes the paper as "a feed-forward neural network that can directly estimate all key 3D scene properties for hundreds of input views".

How many images does VGGT take, and how fast is it?#

The README says VGGT works from "one, a few, or hundreds of its views," and the paper's table shows time rising with the count. With 336 x 518 images on one NVIDIA H100 GPU, 1 frame takes 0.04 seconds and 200 frames take 8.75 seconds. In the paper's table, times stay under one second through 20 frames, and 50 frames take 1.04 seconds.

The post says VGGT works "in less than a second, so very close to real-time." The README says it takes "one, a few, or hundreds of its views."

The paper's Table 9 measures the feature backbone only. It used flash attention v3 and images with "a resolution of 336 x 518."

Input framesTime in seconds (one H100, 336 x 518 images)Peak GPU memory in GB (one H100, 336 x 518 images)
10.041.88
20.052.07
40.072.45
80.113.23
100.143.63
200.315.58
501.0411.41
1003.1221.15
2008.7540.63

On one H100, the paper says the camera head is "lightweight, typically accounting for approximately 5% of the runtime and about 2% of the GPU memory used by the feature backbone."

The README's May 15, 2026 update says "With the same GPU memory budget, VGGT can now run on roughly 2-3x more input frames!" The paper's table, first posted on arXiv on 14 March 2025, predates that update.

The Gradio demo also takes video. In demo_gradio.py, its steps say "Videos will be automatically split into individual frames (one frame per second)." The README warns that "visualizing 3D points may take tens of seconds due to third-party rendering, independent of VGGT's processing time."

The paper's Limitations section says "several limitations remain." It lists three.

  • "the current model does not support fisheye or panoramic images"
  • "reconstruction performance drops under conditions involving extreme input rotations"
  • "although our model handles scenes with minor non-rigid motions, it fails in scenarios involving substantial non-rigid deformation"

Why is this important?#

Previously, if you wanted to create a 3D model of something, you would need to use a 3D scanner (like a Leica BLK360) or something like the LiDAR scanner on an iPhone. While this method works well and can produce excellent results, the high-end hardware is expensive and the process is time-consuming. While there is still work to be done with VGGT, it is just a matter of time until we will be able to generate real 3D models on consumer-grade hardware using just 2D input sources in real-time!

Practical Uses#

Anywhere 3D models are needed, this could be applied. I would wager we'll see this or similar technology introduced on our mobile phones within 12-24 months. Other practical applications of this type of technology could be found in:

  • 3D reconstruction and mapping
  • Computer vision tasks
  • AR/VR from 2D images
  • Autonomous systems
  • Content creation & media
  • Photogrammetry and surveying

Can you use VGGT commercially?#

Only the VGGT-1B-Commercial checkpoint is licensed for commercial usage, as read on November 15, 2026. The README says "only the newly released checkpoint VGGT-1B-Commercial is licensed for commercial usage" and that "the original checkpoint remains non-commercial." LICENSE.txt grants "a non-exclusive, worldwide, non-transferable and royalty-free limited license."

The post says "Anywhere 3D models are needed, this could be applied."

PieceWhat the source says
CodeThe README's July 29, 2025 update says "All code in this repository is now under a commercial-use-friendly license."
License changeThe same update says the license now permits "commercial use (excluding military applications)".
Original checkpointThe Hugging Face card for facebook/VGGT-1B lists the license as "cc-by-nc-4.0".
Commercial checkpointThe README says "Access to the checkpoint requires completing an application form". It also says "The new checkpoint delivers similar performance to the original model."
Terms fileLICENSE.txt is titled "VGGT License" and says "v1 Last Updated: July 29, 2025".
Sharing itLICENSE.txt says "You shall also provide a copy of this Agreement to such third party."
Research papersLICENSE.txt says "you must acknowledge the use of Research Materials in your publication."
Prohibited usesIts Acceptable Use Policy lists "Military, warfare, nuclear industries or applications, espionage" among others.
SupportLICENSE.txt says "Meta is under no obligation to provide any support services for the Research Materials."

The README points to the full terms: "See the LICENSE file for details about the license under which this code is made available."

Demo#

I tested it using just a single photo I took of a LEGO figurine that was near my desk. While the 3D model still has a number of artifacts, I think with either additional images or some post-processing, it would generate a relatively good 3D model. It outputs the results as a glb file that I imported into the Threejs sandbox.

Input Image:

Photo of a Lego minifigure used as the input image

Output Model: (a glb file I loaded in the threejs sandbox)

3D model of the Lego minifigure generated from the photo, loaded in the Three.js editor

How do you run VGGT on your own photos?#

To run VGGT on your own photos, clone the repository, install the requirements, then start the Gradio demo and upload your images. The post says "I tested it using just a single photo I took of a LEGO figurine that was near my desk." The demo needs a CUDA GPU, because demo_gradio.py raises "CUDA is not available. Check your environment." when PyTorch cannot find one.

The README's Quick Start gives the first three commands.

git clone [email protected]:facebookresearch/vggt.git
cd vggt
pip install -r requirements.txt

For the demo, the README then says to run pip install -r requirements_demo.txt and python demo_gradio.py. The repository pins these versions, as read on November 15, 2026.

RequirementWhat the repository says
Pythonpyproject.toml sets requires-python to ">= 3.10".
Core packagesrequirements.txt pins torch==2.3.1, torchvision==0.18.1 and numpy==1.26.1.
Other packagesrequirements.txt lists Pillow, huggingface_hub, einops and safetensors without versions.
Demo packagesrequirements_demo.txt pins gradio==5.17.1 and viser==0.2.23.
WeightsThe README's example says "This will automatically download the model weights the first time it's run, which may take a while."

The demo's own on-screen steps are short.

  1. Upload Your Data. Use the Upload Video or Upload Images buttons on the left "to provide your input."
  2. Preview. "Your uploaded images will appear in the gallery on the left."
  3. Reconstruct. Click the Reconstruct button "to start the 3D reconstruction process."
  4. Visualize. "You can rotate, pan, and zoom to explore the model, and download the GLB file."
  5. Adjust Visualization (Optional). "After reconstruction, you can fine-tune the visualization using the options below".

The demo lists five controls under that optional step.

  • Confidence Threshold. "Adjust the filtering of points based on confidence."
  • Show Points from Frame. "Select specific frames to display in the point cloud."
  • Show Camera. "Toggle the display of estimated camera positions."
  • Filter Sky / Filter Black Background. "Remove sky or black-background points."
  • Select a Prediction Mode. The demo says "Choose between" Depthmap and Camera Branch or Pointmap Branch.

The README says the model "shows surprisingly good performance on single-view reconstruction, although it was never trained for this task." That is the case the post tested with one photo.

It also says that if certain pixels "are unwanted (e.g., reflective surfaces, sky, or water), you can simply mask them by setting the corresponding pixel values to 0 or 1."

The README says you can "try the model with just a few lines of code" through VGGT.from_pretrained with facebook/VGGT-1B. It also says "Alternatively, you can install VGGT as a package." The package page says "you need to install PyTorch and torchvision" first, because "We don't list these as dependencies to avoid CUDA version mismatches."

What files can VGGT export?#

The Gradio demo lets you "download the GLB file," and the README says VGGT supports "exporting VGGT's predictions directly to COLMAP format." The post says "It outputs the results as a glb file that I imported into the Threejs sandbox."

FormatWhat the source says
glbIn demo_gradio.py, the viewer steps say you can "download the GLB file". The script writes the scene to a file ending in .glb.
COLMAPRun python demo_colmap.py --scene_dir=/YOUR/SCENE_DIR/. The README labels this "Feedforward prediction only".
COLMAP with bundle adjustmentAdd --use_ba. The README labels this "With bundle adjustment".
COLMAP with reduced parametersAdd --max_query_pts=2048 and --query_frame_num=5 to the bundle adjustment command. The README flags a trade-off for complex scenes and says "you may consider setting query_frame_num equal to your total number of images".
Gaussian splattingThe README says "The exported COLMAP files can be directly used with gsplat for Gaussian Splatting training" and "we recommend gsplat==1.3.0".

The README says to store the input images in /YOUR/SCENE_DIR/images/ and that "This folder should contain only the images." The result "will be automatically saved under /YOUR/SCENE_DIR/sparse/ in the COLMAP format," as cameras.bin, images.bin and points3D.bin.

The COLMAP script needs extra packages. A comment in requirements_demo.txt says "feel free to skip the dependencies below if you do not need demo_colmap.py," then lists pycolmap==3.10.0 and pyceres==2.3.

Can you train or fine-tune VGGT yourself?#

Yes. The README's July 6, 2025 update says "Training code is now available in the training folder, including an example to finetune VGGT on a custom dataset." The post says "It is very much still in research project mode."

ResourceWhat the source says
Evaluation codeThe README's May 3, 2025 update says "Evaluation code for reproducing our camera pose estimation results on Co3D is now available in the evaluation branch."
Fine-tuningThe paper says its limitations can be addressed "by fine-tuning the model on targeted datasets with minimal architectural modifications."
Smaller modelsThe README's checklist shows "Release VGGT-500M and VGGT-200M" as not yet checked off, as read on November 15, 2026.

Frequently asked questions

What does VGGT stand for?

It comes from the paper title VGGT: Visual Geometry Grounded Transformer. The project provides demo code for that paper.

What possible uses does the author list?

The possibilities include 3D reconstruction and mapping, computer vision, AR/VR from 2D images, autonomous systems, content creation, photogrammetry and surveying.

What happened in the author's test?

A single photo of a LEGO figurine produced a 3D model with artifacts. The result was exported as a glb file and imported into the Threejs sandbox.

What does VGGT predict?

The README says VGGT directly infers extrinsic and intrinsic camera parameters, point maps, depth maps and 3D point tracks. It does this from one, a few or hundreds of views of a scene.

Which award did the VGGT paper win?

CVPR 2025 Best Paper, according to CVPR's awards page. The authors are Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht and David Novotny.

Can I use VGGT commercially?

Only the VGGT-1B-Commercial checkpoint is licensed for commercial usage, according to the README as read on November 15, 2026. It says the original checkpoint remains non-commercial and that access to the commercial one requires completing an application form. LICENSE.txt has the full terms, including its prohibited uses.

How do I export a VGGT scene as a glb or COLMAP file?

Use the Gradio demo for a glb file and demo_colmap.py for COLMAP files. In demo_gradio.py, the viewer steps say you can download the GLB file. The README says demo_colmap.py saves cameras.bin, images.bin and points3D.bin in the sparse folder of your scene directory.

How many images can VGGT use at once?

The README says VGGT works from one, a few or hundreds of views. The paper's largest timing is 200 frames, which took 8.75 seconds and 40.63 GB of peak GPU memory on one NVIDIA H100 GPU with 336 x 518 images, measuring the feature backbone only. The README's May 2026 update says VGGT can "now run on roughly 2-3x more input frames" with the same GPU memory budget.

Stay In The Loop

Never miss an update. Get the latest blogs, webinars, and expert insights straight to your inbox from The Gnar.