VisualSFM : A Visual Structure from Motion System

Changchang Wu  


Basic Usage
    Using the VisualSFM GUI
    Using VisualSFM through command-line
    Dependency on SiftGPU/PBA and PMVS/CMVS
    Typical hardware requirements
    Something to watch out for high resolution images
Advanced Usage
    Specify your pair-list for matching
    Use your own feature matches   
    Use your own feature detectors
    Adjust parameters for higher speed
Technical Details
    The camera model and coordinate system
    Focal length initialization
    The intermediate: features and matches
    The output format: N-View Match (NVM)
Work with the GUI
    Mouse Controls and Navigation
    Keyboard shortcuts
Some Cool Features
    Command Lines within VisualSFM!
    Show animations easily with VisualSFM
    Manual Intervention of reconstruction
Limitations and Issues
    Potential problem with small GPU memory
    Limitations of VisualSFM
    Multi-threading stability
Frequently Asked Questions (FAQs)
    1. The meaning of some typical error messages (e.g. ERROR1).
    2. Where can I download the depending tools?
    3. Dense reconstruction tools other than PMVS?
    4. What if some of the GPU computations do not work?
    5. What if VisualSFM fails to find two images for initialization?
    6. What if VisualSFM produces multiple models?
    7. Is VisualSFM open-sourced? Programmable control of VisualSFM?

3D Reconstruction - Using the VisualSFM GUI
You typically have the following steps:
1. Add the images into your SfM Workspace
   Option 1. "File->Open Multi Images". You can do this multiple times.
   Option 2. "SfM->Load NView Match". Choose a TXT file containing relative image paths.
   Option 3. "SfM->Load NView Match". Choose "All Jpegs in Folder" from format, ant select one.
   Or, use the corresponding toolbar buttons when you see icons on menu items.

2. Run feature detection & full pairwise image matching
   Click "SfM->Pairwise Matching->Compute Missing Match"
   You can go back to step 1, add more images, and compute the missing matches again.
For customized feature detection and feature matching, click here

3. Run sparse reconstruction
   "SfM->Reconstruct Sparse" or the corresponding toolbar button.
   The bundle adjustment step will use Multicore Bundle Adjustment automatically.
4. Dense reconstruction by using Yasutaka Furukawa's CMVS/PMVS (obtain CMVS package yourself!)

   "Sfm->Reconstruct Dense". Note that CMVS/PMVS related parameters are stored in nv.ini
   You will be prompt to save [name].nvm file, and CMVS will run in the folder [name].nvm.cmvs
   If you save the reconstruction to [name].
nvm, [name].i.ply is the result of the i-th model.

*  Save the SfM workspace including 3D reconstruction anytime when idle
   "SfM->Save NView Match". You can open the saved 3D models afterwards.
   It is by default using an
NVM format that contains all the workspace information.

3D Reconstruction - Using Command-line

VisualSFM sfm[options] input output.nvm [user_data_path]
    options = [match_option][sfm_option][misc_option][mvs_option]
    match_option = [+pairs/+import/+subset/+nomatch/]
        +pairs:   match image pairs from [user_data_path];
        +import:  load feature matches from [user_data_path];
        +subset:  match a prioritized subset of pairs when
                  #images >= param_prioritized_subset_switch;
        +nomatch: reconstruction without feature matching;
        default:  compute missing pairwise matches.
    sfm_option = [+resume/+add/+skipsfm/+loadnvm/]
        +resume:  load NVM file, try add new images from
                  [input].txt, and grow the existing models;
        +add:     load NVM file, and find more points and
                  projections for the existing models;
        +loadnvm: load NVM file and skip feature matching;
        +skipsfm: skip sparse and dense reconstruction;
        default:  run regular sparse reconstruction.
    misc_option = [+k=fx,cx,fy,cy/+shared][+sort][+gcp]
        +k:       fixed calibration, e.g. +k=1024,800,1024,600;
        +shared:  enforce shared calibration in the end;
        +sort:    keep the input image order in the output NVM;
        +gcp:     load GCPs from [input].gcp and transform the 3D model.
    mvs_option = [+pmvs/+cmvs/+cmp/]
        +cmvs:    undistort images, run CMVS/genOption, skip PMVS;
        +pmvs:    undistort images, run CMVS/genOption/PMVS;
        +cmp:     undistort images, write p-matrices for CMP-MVS;
        default:  skip the entire dense reconstruction.
    <output.nvm> is where reconstruction is saved.
       You can open it afterwards for visualization.

* Command line examples:
  VisualSFM sfm ../my_jpg_folder ../results/my_result.nvm //sparse reconstruction from folder
  VisualSFM sfm+pmvs my_jpg_list.txt my_result.nvm        //sparse+dense from image list
  VisualSFM sfm+resume initial.nvm final.nvm              //resume sparse reconstruction
  VisualSFM sfm+pairs input output.nvm my_pairs.txt       //use [my_pairs] for matching, and run sfm 
  VisualSFM sfm+pairs input output.nvm @10                //matching pairs within 10, and run sfm 
  VisualSFM sfm+import input output.nvm my_matches.txt    //load [my_matches], and run sfm
  VisualSFM sfm+loadnvm+pmvs sparse.nvm  dense.nvm        //open NVM and run dense reconstruction
  VisualSFM sfm+loadnvm+cmvs input.nvm export.nvm         //open NVM and export PMVS data
  VisualSFM help                                          //show the command line options

* Windows users need to use "cmd /c VisualSFM ..." when using VisualSFM in a batch file.

During the incremental reconstruction step, if you press "Ctrl+C" once, it will quit the loop and save the current model. If you press "Ctrl+C" THREE times, exit(0) will be called.

Using command lines should be slightly faster on large datasets, because it does not generate thumbnails (texture) for visualization. In the GUI mode, you can also skip the pixel loading part by clicking pause "||" button when it says "loading pixel data...".

Dependency on SiftGPU/PBA and PMVS/CMVS
The VisualSFM GUI can run without those libraries. However you need SiftGPU for feature detection and matching, PBA for sparse reconstruction, PMVS/CMVS for dense reconstruction.

SiftGPU & PBA are the GPU-accelerated feature detection and bundle adjustment for the SfM system. (Click here if SiftGPU or PBA doesn't work for you)

is the dense reconstruction module used by VisualSFM. Yasutaka Furukawa's original CMVS is here. Windows binaries can be found in the SfM packages distributed by Pierre Moulon.

* OS-specific instructions are given at the install page.

Typical hardware requirements
1. The feature detection step requires a decent GPU (ATI/ nVidia / Intel)
   In particular, a large amount of GPU memory (1GB) is expected.
   Small GPU memory may cause problems for the feature detection: click here.
   The latest SiftGPU-V400 works for Intel graphic card in some platforms.

2. Multicore bundle adjustment runs on either nVidia CUDA or CPU.

*. GPU computation does not work in Windows remote desktop, use VNC for remote work.
*. If you do not have good graphic cards, click here for the solution.
Something to watch out for high resolution images (when using SiftGPU)
There is an important (modifiable) parameter for SiftGPU feature detection.
        maximum_working_dimension = 3200 by default.

Given an image of size d = max(w, h), Lowe's original SIFT detection starts with the up-sampled image of sz = 2 * d. SiftGPU actually starts the detection with size
     sz' = max{sz/2^i | sz/2^i <= maximum_working_dimension, integer i >= 0};
For example, if your image size is 1024, 1600, 2048, 3200, 4000, the corresponding sizes after the adjustment are 2048, 3200, 2048, 3200, 2000 respectively.

This is usually not a problem. Sometimes you want to get more features for better result, so you may not want size 4000 reduced to 2000. In such cases, you can do the following:

1. If you are sure you have enough memory for size 4000,
    you should change the parameter by "Tools -> Enable GPU -> Set Maximum DIM"

2. Resize your input images to size 3200

In addition, SiftGPU may down-sample again according to the available GPU memory, which may cause undesired behaviors on small GPU memory. Check here.
Specify your pair-list for matching
This allows to match part of the image pairs instead of doing the full-pairwise matching
1. Write a txt file that contains the pairs of image paths; no white space within path!
2. Load your images into VisualSFM;
3. Use "SfM->Pairwise Matching -> Compute Specified Match";
4. You may specify additional list as many times as you like.

*. The same function for command line is option "+pairs"
*. Do NOT use "SfM->Pairwise Matching->Compute Missing Match", which does full matching.

For example, you can compute 3-pair matching with the following file
a.jpg b.jpg
a.jpg c.jpg
b.jpg c.jpg

For simple video sequences, where you want to match every two frames in a certain range, use "SfM->Pairwise matching->Compute Sequence Match". For large datasets, you may use recognition methods (Vocabulary tree or GIST clustering) to find the image pairs for matching.

Use your own feature matches 
1. Write a txt file that contains all the feature matches
2. Load your images (with features) into VisualSFM
3. Use "SfM->Pairwise Matching->Import Feature Matches"
4. You may add feature matches by using the same method again.

*. This assumes that you already have detected the features.
*. The same function for command line is option "+import"
*. Do NOT use "SfM->Pairwise Matching->Compute Missing Match", which does full matching

The match file is in the following format
Match file = <List of Image-Match>
Image-Match = <image1_path> <image2_path> <# of matches>
              <List of 0-based feature indices in image1>
              <List of 0-based feature indices in image2>

For example, the following gives the 24 matches between 888.jpg and 709.jpg
888.jpg 709.jpg 24
19 18 24 3651 1511 2899 71 115 201 202 199 1639 2595 210 189 1355 268 241 137 728 1899 193 192 325
139 143 181 261 342 349 373 433 622 623 686 700 745 812 868 951 987 990 1001 1016 1021 1046 1047 1069

where feature #19 of 888.jpg matches feature #139 of 709.jpg

Make sure the 0-based feature indices are within the correct ranges!

Use your own feature detectors
Option 1. You can write a binary feature detector that works like Lowe's SiftWin32.exe.
          Or you can write a shared library that has a SiftGPU-like interface.

Option 2. Write the .sift files in Lowe's ASCII format, they will be automatically converted to the VisualSFM binary format. (Let me know if there is a bug).

Option 3. Write the .sift files in the binary format as follows:
[Header][Location Data][Descriptor Data][EOF]

[Header] = int[5] = {name, version, npoint, 5, 128};
name = (
'S'+ ('I'<<8)+('F'<<16)+('T'<<24));
version = ('V'+('4'<<8)+('.'<<16)+('0'<<24)); or ('V'+('5'<<8)+('.'<<16)+('0'<<24)) if containing color info
npoint = number of features.

[Location Data]  is a npoint x 5 float matrix and each row  is [x, y, color, scale, orientation].
Write color by casting the float to unsigned char[4]
scale & orientation are only used for visualization, so you can simply write 0 for them

* Sort features in the order of decreasing importance, since VisualSFM may use only part of those features.
* VisualSFM sorts the features in the order of decreasing scales.

[Descriptor Data] is a npoint x 128 unsigned char matrix. Note the feature descriptors are normalized to 512.

 int eof_marker = (0xff+(
Adjust parameters for higher speed
Most parameters are stored in nv.ini, which is automatically generated in the first run.

Feature detection speed
In the GUI mode, use "Menu->Tools->Enable GPU->Customized Param"
In the console mode, use "VisualSFM sfm[+pmvs] input output [SiftGPU parameters]"
You need to check the documentation of SiftGPU on those parameters.
Typically there is no point doing this unless your run out of graphic memory.

Feature matching speed
"param_gpu_match_fmax" is the number of features per-image that are used in feature matching. The default is 8192, but 4k should be enough in most cases. Note that the features are sorted by scale, so the features with largest scales will be used.

If you know the image neighborhood such as adjacent frames in video, you should specify a pair-list for matching instead using the full pair-wise matching.

Lots of time might be spent on saving results (especially for NFS), a beta feature is now enabled by default under Linux to improve the efficiency. It can be disabled by setting param_asynchronous_write= 0 or unchecking "Pairwise Matching->Asynchronous Writes"

Incremental reconstruction speed
"param_bundle_full_iteration" is the number of BA iterations when bundling the full model.
"param_bundle_full_frequency" controls how frequent full bundle adjustment is performed.
These two parameters are what determine the speed of reconstruction.

Dense reconstruction speed
"param_cmvs_max_images":   the cluster size for CMVS, check CMVS documentation
The camera model and coordinate system
Be careful it is slightly different with other SFM software (such as bundler).

The internal camera model has 8 parameters (7 if radial distortion is disabled)
  Given camera K[R T], K = [f, 0 0; 0 f 0; 0 0 1], radial distortion r, and a 3D point X.
  The reprojection in the image is [x, y, z]' = K (RX + T) -> (x/z, y/z)'
  Let the measurement be (mx, my), which is relative to principal point (typically image center)
  The distortion factor is r2 = r * (mx * mx + my * my)
  The undistorted measurement is (1 + r2) * (mx, my)
  Then, the reprojection error is (x/z - (1 + r2) mx, y /z - (1 + r2) my)

NOTE that the parameters saved in NVM file is slightly different with the internal representation. Instead, NVM saves the following for each camera:
  f, R (as quaternion), C = - R'T, rn = r * f * f.
The PBA code includes functions for loading the NVM file and convert the camera parameters.
The radial distortion model is different with other softwares:
* Dan Costin provides an efficient code for undistorting the images under this model.

As for the image coordinate system, X-axis points right, and Y-axis points downward, so Z-axis points forward.

The principal points are assumed to be at image centers except when using a single fixed calibration. When using fixed calibration [fx, cx, fy, cy], the error is estimated in a transformed image coordinate system.
  K = [fx 0 0; 0 fx 0; 0 0 1],  Kc = [1 0 cx; 0 fy/fx cy; 1],
  You can see that Kc * K = [fx 0 cx; 0 fy cy; 0 0 1];

  Let (u, v, 1)' be an original feature location (relative to top-left corner of images)
  The measurement is defined as
(mx, my, 1)' = Inv(Kc) * (u, v, 1)'

Note: the feature locations saved in NVM files are still relative to the image centers rather than the calibrated principal point [cx, cy];

Focal length initialization
The focal lengths (in pixels) are automatically calculated according to EXIF.Focallength and FocalplaneXRes/FocalplaneYRes rather than using a CCD database.

TIP: Keep the original EXIF if you resize JPEGs! (e.g. XNView can do that)

In case the EXIF does not contain those information, the focal lengths are set to be 1.2 * max(width, height), which corresponds to a medium viewing angle.

VisualSFM also support using a single fixed calibration [fx, cx, fy, cy] for all images.
Use menu "SfM -> More functions -> Set Fixed Calibration"

The intermediate: features and matches
VisualSFM automatically saves feature detections and feature matches to disk. There will be [name].sift and [name].mat stored for every image file [name].[ext].  [name].sift stores all the detected SIFT features, and [name].mat stores the feature matches.

TIP: When adding new photos (or close/restart), VisualSFM will match only what is missing.

If you want to change feature-detection parameters, and re-run the reconstruction. You need to delete all the corresponding .sift and .mat files.

If you want to read back the matching results, check the match file code here; or you can use "SfM->Pairwise Matching->Export F-Matrix Matches".

The output format: N-View Match (NVM)
VisualSFM saves SfM workspaces into NVM files, which contain input image paths and multiple 3D models. Below is the format description

NVM_V3 [optional calibration]                        # file version header
<Model1> <Model2> ...                                # multiple reconstructed models
<Empty Model containing the unregistered Images>     # number of camera > 0, but number of points = 0
<0>                                                  # 0 camera to indicate the end of model section
<Some comments describing the PLY section>
<Number of PLY files> <List of indices of models that have associated PLY>

The [optional calibration] exists only if you use "Set Fixed Calibration" Function
FixedK fx cx fy cy

Each reconstructed <model> contains the following
<Number of cameras>   <List of cameras>
<Number of 3D points> <List of points>

The cameras and 3D points are saved in the following format
<Camera> = <File name> <focal length> <quaternion WXYZ> <camera center> <radial distortion> 0
<Point>  = <XYZ> <RGB> <number of measurements> <List of Measurements>
<Measurement> = <Image index> <Feature Index> <xy>

Check the LoadNVM function in util.h of Multicore bundle adjustment code for more details.  The LoadNVM function reads only the first model, and you should repeat to get all. Note the whitespaces in <file name> are replaced by '\"'.

* Use the menu items under Menu->View to choose a view mode or change parameters.
* Quicker switch can be done with mouse, keyboard shortcuts, or toolbar buttons.
* You can save the current view as a JPEG file or copy it into memory (paste to ppt?)
* More documentation is coming
Mouse Controls and Navigation
* Right double click on a camera in 3D point mode to view the selected image
* Left double click on an image in thumbnail mode to view the selected image
* Right double click in the single image mode to return to previous view mode.
* Pan/rotate by Left/Right click + hold + move.
* Zooming by Shift + Mouse, or mouse wheel
* change point size by using "Alt + Mousewheel"
* Change the relative camera size by Ctrl + mouse wheel.
* Just try left/right double/single clicking (on empty space or on camera, on image)

Keyboard Shortcuts
'Z'              zoom fit for 2D Views; reset for 3D views.
'backspace'      return to thumbnail view
'TAB'            switch between dense 3D model and sparse 3D model
'TAB'            switch between original image and its undistorted view
'Pause'          show only points seen by current camera
'T'              Switch between 3D visualization mode: camera+point/camera/point
'S'              switch SIFT display style in single image view mode
'F'              show/hide features in single image view mode
'F'              show/hide camera in Dense reconstruction mode
'A'              show/hide coordinate axis in dense/sparse reconstruction
'left/right'     switch to previous/next image/pair...
'up/down'        switch to previous/next 3D model

Command Lines within VisualSFM
Hit ENTER in the main window, a command textbox will show up in the statusbar.
"am <cmd>"           Animation manager
"log"                Use default text editor to open log file
"clog <filename>"    Replace the default log file
"locate"             Open the current image in File Explorer
"pwd"                Print out current directory
"cd <path>"          Change current Directory
"skipm"              Do not load matching records when loading NVM
"ci"                 Print out information on current camera
"pi"                 Print out information on current pair
"mi"                 Print out information on current model
"psz <float>"        Change the default point size for OpenGL
"czoom <int>"        Change the zoom factor for high-res screen capture
"ao"                 Adjust coordinate system to fit the dominant camera plane
"sort"               Re-order the cameras to the input order (applies to all models).
TAB                  Show previous commands

Show animations easily with VisualSFM
Add current view to animation list by clicking F4. After adding several views (different viewpoints or different view modes), click F5 once and you will see the transition animation of the saved views. Double-click F5, the animation will play in a loop. This is how I make the videos for my papers.

Type "am" in the internal command line, and you will see the full list of commands for making animations.

Manual Intervention of reconstruction
In the GUI mode, you can pause/resume the incremental reconstruction.
Look for the following buttons: >>+ Resume, || Pause

In case there are incorrectly registered cameras, you can delete them
* Select a camera by a right clicking on the camera in the 3D N-View Points Mode
* Delete the camera from the 3D model by clicking the (hand) button

You can also manually choose the initialization pair, see FAQ below.

Potential problem with small GPU memory
By default, SiftGPU tries to fit your memory cap by down-sampling the images when needed. It is possible that you get much less features on small-memory GPUs, which produces different results on different machines.

You can check the remote/CPU solutions in FAQ, or

You can also try disable the down-sampling and re-run experiments.
   1, delete all the .sift and .mat files
   2, choose "Tools -> Enable GPU -> Customized Param"
      Add "-nomc" to the parameter list, which means "no memory cap"
   3, re-run matching and reconstruction.
   *, this does not guarantee to work because of the out-of-memory possibility.
   *, this may work, and may become extremely slow because of using virtual memory.
Limitations of VisualSFM
1. The keypoint-based reconstruction only works with textured surfaces.
   - This tool will not work if too few features are detected
   - e.g. white wall, uniform color objects

2. The incremental reconstruction method has many limitations:
   - loops are not always closed,
   - sensitive to initialization (you may need to try manual initialization)
   - Unstable for degenerate motion such as forward motion
   - Not utilizing the continuity of motion for videos (you may look into SLAM tools).

3. There is only one radial parameter. It is sufficient for commodity photos, but
   may not work if your camera has large distortions that do not fit the model.

4. The 32-bit versions may run out memory easily on large datasets due to OS limitation.

Multi-threading stability
During the reconstruction process, the points and cameras are being modified, while they are also being visualized. Since I do not want the reconstruction get slowed down by the visualization, very limited mutex control is used.

Some memory management has been done to ensure it is very unlikely to crash when you view the live reconstruction (rotating, zooming, clicking, ...). But there is still a very very small chance of memory leak.

This is not a problem for the console mode. Similarly, if you simply minimize your GUI window or avoid too much mouse dragging, the chances of such memory leak will be reduced.

FAQ-1. The meaning of some typical error messages (e.g. ERROR1).

1. "SiftGPU failed the detection test" or "ERROR: siftgpu doesn't work"
    This means SiftGPU feature detection did not work properly. Either your graphic driver
    is out-dated or your graphic card does not support SiftGPU at all. You should give
    it another try after updating the graphic driver. If it fails the same way, you may
    need to switch to CPU for feature detection or get a better GPU.

2. "ERROR1" + "ERROR: unable to locate sift binary"
    This means VisualSFM tried to use CPU feature detector but it could not find it.
    Under windows, it looks for .\SiftWin32.exe if param_use_vlfeat_or_lowe==0 or
    .\sift.exe otherwise. For other OS, it will look for ./sift

3. "CMVS cannot be located!" or "ERROR: the above CMVS command failed!\"
    This means VisualSFM failed to launch cmvs, genOption, or pmvs2.

    * Usually this happens if you do not have the CMVS binaries in the right place.
        VisualSFM[.exe] expects in the same folder:
cmvs.exe/pmvs2.exe/genOption.exe [pthreadVC2.dll] for Windows,
          -- cmvs/pmvs2/genOption for other OS.
        See FAQ-2 if you haven't downloaded them yet.

    * Note these commands may also fail due to bad input or their own bug. You should
      try run these commands yourself from your system terminal and look at the
      original log from these binaries.

FAQ-2. Where can I download the depending tools.
1. SiftGPU & PBA (multicore bundle adjustment)

2. CPU feature detectors (set param_use_siftgpu to 0 to use them)
    Lowe's SIFT : siftWin32.exe or sift (set param_use_vlfeat_or_lowe to 0)
    VLFeat SIFT : sift + vl.dll/ (set param_use_vlfeat_or_lowe to 1)

    Compile from the original source
    Windows binaries by Pierre Moulon
             cmvs.exe/pmvs2.exe/genOption.exe and pthreadVC2.dll
FAQ-3. Dense reconstruction tools other than PMVS?
VisualSFM natively supports running the CMVS/genOption/PMVS2 tool chain, but you need to run Meshlab to convert the dense points to mesh.

Several additional softwares work with VisualSFM:

CMP-MVS by Michal Jancosek: this tool produces meshes directly. You need to click VisualSFM's 'Reconstruct Dense' function and select 'NVM->CMP-MVS' to convert the data.

SURE by Mathias Rothermel and Konrad Wenzel: SURE natively support the NVM file format, and you do not need to run the export function.

MVE by Michael Goesele's research group: this tool works with the same format of exported data as CMVS/PMVS

MeshRecon by Zhuoliang Kang: this tool produces meshes directly and works with the NVM file (no radial distortion, Windows only).
FAQ-4. What if some of the GPU computations do not work?

CPU replacement for feature detection (No needed for most cards, even Intel)
1. Obtain SIFT binary(Lowe's of VLFeat)
2. Put the SIFT binary in the same directory.
3. Choose Lowe's binary of VLFeat binary by setting param_use_vlfeat_or_lowe
4. Disable GPU feature detection by "Tools->Enable GPU->Disable SiftGPU"
   or modify parameter param_use_siftgpu to 0 for permanent selection

Remote solution:
1. Run VisualSFM on a CUDA-capable Linux server through terminal (X-terminal for GUI).
2. You may copy the workspace back to local machine for visualization.

Using CPU for feature matching (No needed for most cards, even Intel)
Disable GPU feature matching by unchecking "Tools->Enable GPU->Match Using GLSL"
*. modify parameter param_use_siftmatchgpu to 0 for permanent selection

Using CPU for multicore bundle adjustment (GPU-based PBA does not supporting ATI)
Windows: Use the non-CUDA packages.

Linux: Download PBA code, run make with makefile_no_gpu, and rename
       to, and copy it to the VisualSFM bin folder.

FAQ-5. What if VisualSFM fails to find two images for initialization?
In case VisualSFM does not find a good pair to initialize, you can try manually choose it.
Option 1. Switch to "2-View 3D points", use left/right to scan through all image pairs, find a pair with 3D points, choose "SfM->More Functions->Set Initialization Pair".

Option 2. If you know which two images are good, right click the two images in the "Thumbnail View", verify it in "2-View 3D points", and choose "Set Initialization Pair".
FAQ-6. What if VisualSFM produces multiple models?
First, check if feature detection and matching are working correctly. Basically, you check the number of detected features and matches, and see if they are too small.

Second, if feature detection and matching works correctly, there are two different cases

The common case: two models do not share enough images. The problem is caused by the lack of feature matches between very different viewpoints. You should get more pictures to cover the transiting viewpoints.

The rare case: two models share enough images (param_image_reuse_max). Try use "SfM->More Functions->Merge Sparse Models" (V0.5.26+), and see if it merges the models. Such problems are often caused by the choice of initialization. You may delete one model, and try resume the reconstruction of the kept model. 

FAQ-7. Is VisualSFM open sourced? Programmable control of VisualSFM?
The GUI (including SfM) part of the software package is close-sourced, which I developed from scratch.  If you are interested in the algorithm detail of the SfM system, you can read my 3D paper here.

However, I have already open-sourced SiftGPU (feature detection & matching) and PBA (multicore bundle adjustment), which can be easily integrated into any other SfM system. You can also make changes to them, and reintegrate with VisualSFM.

If you want some simple programmable control of the SfM system, you can either use VisualSFM through command line, or use VisualSFM as a server through socket command interface (Python wrapper by Nick Rhinehart).