ASAP

This repository contains the official code and dataset of ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility (ICRA 2024).

Authors: Yunsheng Tian, Karl D.D. Willis, Bassel Al Omari, Jieliang Luo, Pingchuan Ma, Yichen Li, Farhad Javid, Edward Gu, Joshua Jacob, Shinjiro Sueda, Hui Li, Sachin Chitta, Wojciech Matusik

Summary: The automated assembly of complex products requires a system that can automatically plan a physically feasible sequence of actions for assembling many parts together. In this paper, we present ASAP, a physics-based planning approach for automatically generating such a sequence for general-shaped assemblies. ASAP accounts for gravity to design a sequence where each sub-assembly is physically stable with a limited number of parts being held and a support surface. We apply efficient tree search algorithms to reduce the combinatorial complexity of determining such an assembly sequence. The search can be guided by either geometric heuristics or graph neural networks trained on data with simulation labels. Finally, we show the superior performance of ASAP at generating physically realistic assembly sequence plans on a large dataset of hundreds of complex product assemblies. We further demonstrate the applicability of ASAP on both simulation and real-world robotic setups.

Installation

1. Clone repository

git clone git@github.com:yunshengtian/ASAP.git

2. Python environment

Step 1:

conda env create -f environment.yml
conda activate asap

or

pip install numpy networkx matplotlib scipy pyglet rtree sortedcontainers scipy tqdm trimesh torch torch_geometric seaborn ikpy pyquaternion

Step 2:

Due to dependency issues, the libraries below must be installed after the above step is complete.

pip install torchsparse torchscatter

3. Python binding of simulation

cd simulation
python setup.py install

To test if the installation steps are successful, run:

python testsim/testsimplesim.py --model box/boxstack --steps 2000

We also provide a beam assembly under `assets/beam_assembly folder. To visualize the simulation of that, run:

For point-based SDF collision check to work more accurately, we highly recommend subdividing the assembly meshes to have denser contact points by running assets/subdividebatch.py. For example, to subdivide the dataset saved in assets/testassembly and export to assets/testassemblydense:

Important arguments include (see the complete list in plansequence/runseq_plan.py):

• dir: assembly directory (relative to assets/)
• id: assembly id
• planner: name of the node selection algorithm (i.e., tree search planner) (see plan_sequence/planner/init.py for supported options)
• generator: name of the part selection algorithm (i.e., part generator) (see plan_sequence/generator/init.py for supported options)
• seed: random seed
• budget: maximum number of feasibility evaluation
• max-gripper: number of available grippers (for assembling and holding parts)
• max-pose: number of pose candidates to search from during pose selection
• pose-reuse: number of poses to be reused from the parent node for pose selection
• early-term: early termination once a feasible plan is found (rather than waiting for the whole tree to be fully expanded)
• timeout: timeout in seconds for the whole sequence planning
• base-part: id of the base part (if exists) as the first part that stays in place (reorientation will not be allowed then)
• log-dir: log directory for storing all the planning outputs
• plan-grasp: whether to plan gripper grasps
• plan-arm: whether to plan arm motions

If log-dir is specified in the above command, the log files will be saved in this directory: {log-dir}/{planner}-{generator}/s{seed}/{id}/.

• setup.json that stores the arguments used for experiments;
• stats.json that stores the high-level planning results;
• tree.pkl that stores the explored disassembly tree with all necessary information on edges/nodes.

Important arguments include (see the complete list in plansequence/playlogged_plan.py):

• log-dir: input log directory
• assembly-dir: input assembly directory (absolute path)
• result-dir: output result directory
• save-mesh: whether to output meshes in the result folder (not necessarily needed, same as meshes in assembly dir)
• save-pose: whether to output (reoriented) pose of every assembly step
• save-part: whether to output parts to be held
• save-record: whether to output rendered videos
• save-all: whether to output everything above
• reverse: whether to reverse the rendering (to be assembly instead of disassembly)
• show-fix: whether to show fixed parts in rendering (in grey)
• show-grasp: whether to show gripper grasp in rendering
• show-arm: whether to show arm motion in rendering
• camera-lookat and camera-pos: camera view parameters (to find the best view for rendering, you can do that by pressing V in simulation viewer when testing the simulation for your assembly, also see the below Simulation Viewer section)

After the animations are generated, you can use plansequence/combineanimation.py to concatenate all videos into a single one.

Use plansequence/runseqplanbatch.py to run batch sequence planning for all assemblies in the assembly directory (with similar arguments as shown above for the serial script). The log folders will be saved in this directory: {log-dir}/g{max-gripper}/{planner}-{generator}/s{seed}/.

• Scroll your mouse wheel to zoom in/out.
• Move your mouse while holding down the left mouse button to rotate the camera.
• Move your mouse while holding down Shift + left mouse button to move around the scene.
• Press S to slow down and F to speed up the simulation.
• Press Space to pause/resume the simulation.
• Press V for outputting the camera parameters (lookat and pos).

We provide a pre-processing script assets/processmesh.py that rescales the custom meshes to provide an unified input to the algorithm. Assume you have a set of .obj meshes placed under sourcedir/ that form an assembly and are all in the assembled states. Run:

Then, all the pre-processed meshes will be written to the output directory target_dir/.

To run our algorithm on top of them, just specify --dir and --id accordingly for the scripts in the above Experiments section to make sure they can find targetdir/`.

Contact

Please feel free to contact yunsheng@csail.mit.edu or create a GitHub issue for any questions about the code or dataset.

Citation

If you find our paper, code or dataset is useful, please consider citing:

@article{tian2023asap,
  title={ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility},
  author={Tian, Yunsheng and Willis, Karl DD and Omari, Bassel Al and Luo, Jieliang and Ma, Pingchuan and Li, Yichen and Javid, Farhad and Gu, Edward and Jacob, Joshua and Sueda, Shinjiro and others},
  journal={arXiv preprint arXiv:2309.16909},
  year={2023}
}