对于机械臂视觉抓取功能来说，比较传统的做法：由于之前的物体识别模块已经确定了这片点云对应的物体，如果我们有物体的3D模型，便可以直接用ICP算法将这片点云与物体的3D模型对齐。既然知道3D模型的位姿那就好办了。之后就是各种抓取姿态生成、力封闭之类的东西了。其他方法：利用RCNN在图像中定位了物体的大概位置，然后根据一堆点云做抓取，而不用知道物体的3D模型。 1、Using Geometry to Detect Grasp Poses in 3D Point Clouds 上面这个是ICRA 2015中的一篇论文，他们利用SVM分类器，直接对3D点云进行训练，判断其可抓取方向：感觉效果并不是特别好，需要较为完整的点云，同时训练库对结果影响也很大。 2、High precision grasp pose detection in dense clutter 这篇论文采用卷积神经网络训练一小片点云的可抓取性，上图按顺序依次是：彩色图片、点云图片、随机生成的抓取姿态、CNN判断可行的抓取姿态。据作者说抓取成功率相比于第一种方法提升了20%。 这两种方法，作者都已经将程序开源出来了，第一种方法是agile_grasp，在ROS中编译安装很简单，第二种方法是gpd_ros，这个包安装编译就有很多坑，首先gpd_ros依赖于gpd库，而作者提供的gpd2.0.0版本是基于c++14和PCL1.9的版本写的，所以对于ROS kinetic用户来说配置起来错误很多。最终要想编译gpd_ros成功，需要以下几个步骤：
- 下载编译PCL1.9
- 修改cmake版本
- 下载编译gpd
- 添加opencv依赖
- 编译gpd_ros
1. ROS环境下载编译PCL1.9
https://blog.csdn.net/czychen1997/article/details/107306674/?utm_medium=distribute.pc_relevant.none-task-blog-utm_term-3&spm=1001.2101.3001.4242 现在在官网好像没法下载源码了，要到github下载才行:https://github.com/PointCloudLibrary/pcl/releases/tag/pcl-1.9.0 ,下载源码之后，放到你想要的安装目录下，然后安装依赖
$ sudo apt-get update
$ sudo apt-get install git build-essential linux-libc-dev
$ sudo apt-get install cmake cmake-gui
$ sudo apt-get install libusb-1.0-0-dev libusb-dev libudev-dev
$ sudo apt-get install mpi-default-dev openmpi-bin openmpi-common
$ sudo apt-get install libflann1.8 libflann-dev
$ sudo apt-get install libeigen3-dev
$ sudo apt-get install libboost-all-dev
$ sudo apt-get install libvtk5.10-qt4 libvtk5.10 libvtk5-dev
$ sudo apt-get install libqhull* libgtest-dev
$ sudo apt-get install freeglut3-dev pkg-config
$ sudo apt-get install libxmu-dev libxi-dev
$ sudo apt-get install mono-complete
$ sudo apt-get install qt-sdk openjdk-8-jdk openjdk-8-jre
然后进入目录编译：
$ cd pcl_1.9
$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_PREFIX=/usr \ -DBUILD_GPU=ON-DBUILD_apps=ON -DBUILD_examples=ON \ -DCMAKE_INSTALL_PREFIX=/usr ..
$ make -j
编译过程中，由于系统的VTK版本比较低，可能会出现以下warning像下面这样的，不用管：
In file included from /home/redwall/catkin_ws/pcl-1.9.0/visualization/include/pcl/visualization/impl/pcl_visualizer.hpp:48:0,
from /home/redwall/catkin_ws/pcl-1.9.0/visualization/include/pcl/visualization/pcl_visualizer.h:2352,
from /home/redwall/catkin_ws/pcl-1.9.0/examples/segmentation/example_cpc_segmentation.cpp:49:
/usr/include/vtk-6.2/vtkMath.h:672:3: warning: multi-line comment [-Wcomment]
// a & b \\
^
/usr/include/vtk-6.2/vtkMath.h:676:3: warning: multi-line comment [-Wcomment]
// 1 & 0 \\
^
/usr/include/vtk-6.2/vtkMath.h:679:3: warning: multi-line comment [-Wcomment]
// a & b \\
等到编译完成之后，虽然有很多警告，只要能编译通过，就没关系，安装即可：
$ sudo make install
此时PCL1.9版本和之前ROS集成的PCL版本是可以共用的，通过在CMakeLists中即可指定特定版本。 期间出现警告可以不管，但出现error时会报错停止，下面是我遇到的一个错误：pcl_visualizer.cpp:3567:88
visualization/src/pcl_visualizer.cpp:3567:88: error: invalid conversion from ‘int’ to ‘const char*’ [-fpermissive]
通过github找到了解决办法：修改源码中pcl_visualizer.cpp，从3563行开始修改成如下代码
#if VTK_MAJOR_VERSION >= 8 && VTK_MINOR_VERSION >= 2
mapper->MapDataArrayToMultiTextureAttribute(mesh.tex_materials[tex_id].tex_name.c_str(),
this_coordinates_name.c_str(),
vtkDataObject::FIELD_ASSOCIATION_POINTS);
#else
mapper->MapDataArrayToMultiTextureAttribute(tu,
this_coordinates_name.c_str (),
vtkDataObject::FIELD_ASSOCIATION_POINTS);
this_coordinates_name.c_str(),
vtkDataObject::FIELD_ASSOCIATION_POINTS);
#endif
polydata->GetPointData ()->AddArray (coordinates);
actor->GetProperty ()->SetTexture (tu, texture);
++tex_id;
}
2. 下载编译GPD库
https://github.com/atenpas/gpd/tree/2.0.0，和上面的方法类似，下载文件之后解压到安装的目录下，但是由于这个包的依赖比较新，编译的时候提示cmake版本低，网上搜索发现，升级cmake需要先卸载老版本的cmake，但是ROS用户会由于这个操作把ROS给干崩，巨坑，所以我只修改了CMakeList.txt，将cmake_minimum_required(VERSION 3.5.1 FATAL_ERROR)中的版本换成自己的较低的版本，实测成功。可以看到后面库要求的依赖是 find_package(PCL 1.9 REQUIRED)是1.9的版本：老规矩先编译在安装
$ cd gpd
$ mkdir build && cd build
$ cmake ..
$ make -j
$ sudo make install
3. 下载编译gpd_ros
https://github.com/atenpas/gpd_ros.git ,编译过程中还是有问题：
In file included from /home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_messages.cpp:1:0:
/home/redwall/catkin_ws/src/grasps/gpd_ros/include/gpd_ros/grasp_messages.h:44:12: error: ‘GraspConfigList’ in namespace ‘gpd_ros’ does not name a type
gpd_ros::GraspConfigList createGraspListMsg(const std::vector<std::unique_ptr<gpd::candidate::Hand>>& hands, const std_msgs::Header& header);
^
/home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_messages.cpp:3:10: error: ‘GraspConfigList’ in namespace ‘gpd_ros’ does not name a type
gpd_ros::GraspConfigList GraspMessages::createGraspListMsg(const std::vector<std::unique_ptr<gpd::candidate::Hand>>& hands, const std_msgs::Header& header)
通过修改gpd_ros包中的CMakeLists.txt，将PCL依赖部分换成find_package(PCL 1.9.0 REQUIRED)，指定成最新升级的版本而不是用老版本。 第二个错误：
[ 95%] Built target gpd_ros_generate_messages
In file included from /usr/local/include/gpd/descriptor/image_generator.h:54:0,
from /usr/local/include/gpd/grasp_detector.h:51,
from /home/redwall/catkin_ws/src/grasps/gpd_ros/include/gpd_ros/grasp_detection_node.h:55,
from /home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_detection_node.cpp:1:
/usr/local/include/gpd/descriptor/image_strategy.h:40:31: fatal error: opencv2/imgproc.hpp: 没有那个文件或目录
compilation terminated.
grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/build.make:62: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/src/gpd_ros/grasp_detection_node.cpp.o' failed
make[2]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/src/gpd_ros/grasp_detection_node.cpp.o] Error 1
CMakeFiles/Makefile2:1341: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/all' failed
make[1]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/all] Error 2
make[1]: *** 正在等待未完成的任务....
In file included from /usr/local/include/gpd/descriptor/image_generator.h:54:0,
from /usr/local/include/gpd/grasp_detector.h:51,
from /home/redwall/catkin_ws/src/grasps/gpd_ros/include/gpd_ros/grasp_detection_server.h:46,
from /home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_detection_server.cpp:1:
/usr/local/include/gpd/descriptor/image_strategy.h:40:31: fatal error: opencv2/imgproc.hpp: 没有那个文件或目录
compilation terminated.
grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/build.make:62: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/src/gpd_ros/grasp_detection_server.cpp.o' failed
make[2]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/src/gpd_ros/grasp_detection_server.cpp.o] Error 1
CMakeFiles/Makefile2:200: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/all' failed
make[1]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j4 -l4" failed
看到了/usr/local/include/gpd/descriptor/image_strategy.h:40:31: fatal error: opencv2/imgproc.hpp: 没有那个文件或目录，应该是opencv库的问题，看了以下CMakeLists.txt，发现果然没有opencv的依赖，添加然后在编译即可：
# Opencv
find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
link_directories(${OpenCV_LIBRARY_DIRS})
add_definitions(${OpenCV_DEFINITIONS})
今天又编译了一次，出现了新的问题：
fatal error: gpd_ros/GraspConfig.h: 没有那个文件或目录
可能是因为.msg文件没来得及生成，多尝试两次即可。
fatal error: pcl_conversions/pcl_conversions.h: 没有那个文件或目录
PCL17才有的头文件，可能安装新版本没有了，可以自己手动安装。 sudo apt-get install ros-kinetic-pcl-conversions 编译，完美通过
分割线
之前配置PCL1.9的时候，对于低版本的VTK，编译过程中会出现一些警告，刚好这段时间用的包需要依赖VTK8.2的版本，于是下载了源码包编译安装，一段时间之后，再次编译gpd_ros出现了一些错误，一番搜索之后，发现好像是更新VTK版本之后，之前编译的动态库就被替换了，换言之就是先基于VTK低版本编译安装了PCL1.9和gpd_ros包，然后升级了VTK8.2版本之后导致PCL找不到原本的动态库，导致gpd_ros编译报错。。。。坑！！！ 由于是通过make源码编译安装的，没办法直接卸载VTK高版本，只能手动删除了,通过make编译的库一般都在/usr/local/lib目录下，具体再那个文件夹下，需要再/usr/local下的几个文件夹仔细找找，一般都是libvtkxxx.so.8.2这种格式的。 删除之后，再次编译gpd_ros之后，就没有上面的错误了，但是新的错误又来了，纠结了好久，千万记得将ROS工作空间里面的build文件夹和devel文件夹清理以下，否则都是之前的编译完成的东西，一旦链接的还是原来的动态库自然就会报错！ 解决pip安装的绝大数问题：
sudo pip install "numba==0.34" --default-timeout=1000 --no-cache-dir -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install "rtree>=0.8,<0.9“ --default-timeout=1000 --no-cache-dir -i https://pypi.tuna.tsinghua.edu.cn/simple
