Bin Picking Tutorial: Setting up Photoneo Bin Picking Studio with UR robots

From 3D scanning Knowledge base - Photoneo wiki
Jump to: navigation, search

NOTE: Users are strongly recommended to read the general introduction to robot interfaces prior to installing specific robot modules.

1. Prerequisities

Photoneo UR Interface is compatible with Universal robots versions 3.4 and higher.

2. UR Controller setup

The following tutorial provides a step by step guide to configure your UR controller and install all necessary UR scripts to get the robot interface for Photoneo Binpicking Studio up and running.

This tutorial was originally written using the latest UR v.3.5.1, however it is compatible with all versions up to UR v.5.2 and with minor changes should be also compatible with older versions. Users are assumed basic knowledge of the UR system and its configuration. If you have never used this robot before, it is strongly recommended that your read the basic [tutorials] on the Zacobria robot community forum for UR robots prior to installing the Photoneo UR interface.

2.1 Network configuration

The first step is to configure the network interface. Click Setup Robot -> Network to open the configuration window:

Alt text
Figure 2.1

The following configuration pane should appear. Select the static address network method and configure the IP settings to meet your network requirements. Click Apply when finished.

Alt text
Figure 2.2

3. Install Photoneo program

The next step is to copy UR scripts from the Photoneo UR Module archive to the robot controller. You should receive this archive (.zip) file as a part of Binpicking Studio. If you havn't received this file, please contact your supplier.

It is possible to copy files from Photoneo UR Module archive to robot controller via Ethernet or using the USB stick directly. In this tutorial both options will be covered:

3.1 Copy files from USB stick

Insert the USB stick into the slot on the back of the pendant. On the Main Robot screen, select Program Robot -> Load Program:

Alt text
Figure 3.1.1

You should see the robot filesystem pane. Select USB -> Photoneo and choose main_module.urp

Alt text
Figure 3.1.2

Click on File -> Save as to save scripts to the robot controller

Repeat the same sequence for calibration.urp

NOTE: If a compatibility error occurs during program loading, it is recommended to create a new program first. See section 3.3 for further information.

3.2 Copy files via Ethernet SCP

It is also possible to copy files to the UR Controller using remote access directly from your PC. Browse to the folder containing Photoneo UR scripts and copy the whole folder using the following command:

$ scp -r Photoneo root@XXX.XXX.XXX.XXX:/programs/Photoneo

The default password is "easybot"

3.3 Creating new program

On the Main Menu screen select Empty Robot. The following screen should appear:

Alt text
Figure 3.3.1

The Photoneo UR module consists of two main parts: BeforeStart and Robot Program. In order to enable BeforeStart Sequence it is necessary to check this option at the bottom of this pane.

You should see the following screen with empty Robot Program and BeforeStart programs now:

Alt text
Figure 3.3.2

Copy the scripts from Photoneo/scripts to the BeforeStart sequence using the following steps:

1. Click BeforeStart on the left panel

2. Select Structure in the Main Window

3. Choose Advanced tab

4. Select Script code

Alt text
Figure 3.3.3

5. Select the Command tab in the Main Window

6. Choose File and Edit as shown in the picture below:

Alt text
Figure 3.3.4

8. Click Open and select photoneo_common.script from the Photoneo/script folder:

Alt text
Figure 3.3.5

10. Hit Save and Exit:

Alt text
Figure 3.3.6

If the script has been properly loaded, it should now be visible under the BeforeStart program as shown in the picture below:

Alt text
Figure 3.3.7

Repeat the same sequence for customer_definitions.script. The pane should now look as follows:

Alt text
Figure 3.3.8

You can also add subprogram error_handling that deals with errors should they appear during binpicking. If an error occurs while calling binpicking procedures flag PHO_OCCURRED_ERR is set and subprogram error_handling should be called. In the picture below the call is placed after sending request for initialization.

NOTE: Error handling can be adapted by the user to better suit his needs.

Alt text
Figure 3.3.9

To call a subprogram you need to load it first. Go to Structure -> Advanced -> SubProg

Alt text
Figure 3.3.10

Then double click on "Call":

Alt text
Figure 3.3.11

Select "Create New". Then choose option "Load File".

Alt text
Figure 3.3.12

Choose file error_handling.urp:

Alt text
Figure 3.3.13

Next time when you need to call this subprogram just go to:

Structure -> Advanced -> SubProg

Double click on "Call" just like the first time, but this time choose "error_handling" from the drop-down menu instead of "Create New".

At this point your Robot Controller is configured to work with Photoneo Binpicking Studio. However your UR script code must be adapted to meet your application requirements - you need to reteach HOME, START and END poses, update part placing, application logic, etc. The following section of the tutorial provides basic examples of how this can be programmed as well as a detailed explanation of Photoneo UR API.

4 Photoneo UR API

The Photoneo UR interface was designed to be easily integrated into existing application written in ur script code. It provides users with two options for determining the final robot behavior:

- UR API - Module photoneo_common.script contains UR requests (procedures) used to control the bin picking sequence. Requests are used within program binpicking.urp.

- UR CONFIG - Module customer_definitions.script contains an application specific settings such as gripper commands, motion speed, precision and overall bin picking sequence definition.

4.1 UR API

Module photoneo_common.script contains API calls (requests) provided by Photoneo UR Interface. These requests are intended for high level control of bin picking sequences and are usually called in program binpicking.urp.

pho_wait_for_server (server_ip, port = PHO_BINPICKING_PORT_DEFAULT, max_attempt = 1) - function to establish a connection to the Vision Controller. Blocking function; suspends the program until a proper connection is established. If the server on the Vision Controller side is not running or does not respond, an error is raised and the UR program is terminated. We recommend ommiting optional parameter port (default value will be used). If you need to use specific port value please contact to help you with configuring port in Binpicking Studio.

pho_request_init (pho_start_pose, pho_end_pose, vision_sys_id = PHO_VISION_ID_DEFAULT, timeout = -1) -request to initialize binpicking application from the Vision Controller side. Start and End Poses defined by the operator from the UR side are transferred to the Vision Controller and used in the trajectory planning pipeline as start and terminus points.

pho_request_scan (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to trigger next scan and localization. Non-blocking request, the UR program continues immediately at next line.

pho_wait_for_scan_completion () - function to wait for scan to be finished. Should be called after triggering the scan, but not immediately.

pho_request_trajectory (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to start trajectory planning for current goal. Non blocking function, the UR program continues immediately at the next line.

pho_receive_trajectory (timeout = -1) - function to receive requested bin picking trajectories (consisting of a sequence of trajectory and gripper operations). Blocking function; waits for motion data to be received. If called without previously calling trajectory_request(), the UR program is terminated.

pho_request_calib_start (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to start calibration. Currently unsupported. Prepared for future use.

pho_request_calib_add_point (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to add calibration point - a scan is triggered and the calibration is recalculated. Blocking request; the program will not continue until the calibration result is recalculated and has been received by the Robot Controller. Currently unsupported. Prepared for future use.

pho_request_calib_set (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to set current calibration result to the PhoXi Scanner. Currently unsupported. Prepared for future use.

pho_request_calib_reset (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to reset current calibration and remove all previously added points. Currently unsupported. Prepared for future use.

pho_other_req (data, size, vision_sys_id = PHO_VISION_ID_DEFAULT) - request used to send additional data to Vision Controller.

pho_request_bin_locator (vision_sys_id = PHO_VISION_ID_DEFAULT) - request to start localization of the bin containing parts to be picked. Bin CAD model position is adjusted in environment. Currently unsupported. Prepared for future use.

pho_request_send_pick_failed (vision_sys_id = PHO_VISION_ID_DEFAULT) - notify Vision Controller about failed pick to lower preference of the part during object-to-pick selection process.

pho_request_change_solution (solution_id) - request to change deployed solution. Currently unsupported. Prepared for future use.

NOTE: Requests with optional parameter timeout have its default value set to -1. This means the timeout for receiving packets is infinite. Since UR script has no means for detection of closing socket the robot program may be left hanging on this function when this situation occurs.


Module customer_definitions.script enables the user to configure several bin picking settings, including gripper command implementations, trajectory speed and precision adjustments.

- open_gripper() - implement function for opening your gripper here

- close_gripper() - implement function for closing your gripper here

- gripper_user_1() - implement custom user gripper command (reserved for future use)

- gripper_user_2() - implement custom user gripper command (reserved for future use)

- gripper_user_3() - implement custom user gripper command (reserved for future use)

- gripper_user_4() - implement custom user gripper command (reserved for future use)

- gripper_user_5() - implement custom user gripper command (reserved for future use)

- gripper_user_6() - implement custom user gripper command (reserved for future use)

- binpicking_settings() - specify speed and precision settings for bin picking trajectories. Adopt number of trajectories to meet your bin picking setup. (by default 4 trajectories are used)

- pho_bin_picking_simple() - the very basic bin picking sequence (for experimental purposes)

- pho_bin_picking() - - executes standard bin picking sequence which consist of various number of operations to be performed. Blocking function; the program does not continue until the whole sequence has finished. Always make sure that the robot is in pho_start pose before calling this function. Argument pho_start_position is a precaution in case the user forgets to move robot to pho_start (DO NOT EDIT!)


Photoneo BP Studip provides basic error handling. If an error occurs during bin picking operation PHO_OCCURED_ERR flag is set to true. PHO_ERR_CODE variable contains code of specific error, which helps to identify the source of the problem.

PHO_NO_ERR = 0 - Service response from Vision Controller is valid

PHO_SERVICE_ERR = 1 - Service response from Vision Controller is invalid.

PHO_UNKNOWN_REQ = 2 - The Vision Controller received an unknown request.

PHO_COM_FAILURE = 3 - Communication failure due to socket closure.

PHO_BAD_DATA = 4 - Data validation check has failed.

PHO_TIMEOUT = 5 - Communication failure due to socket timeout.

PHO_LONG_TRAJECTORY = 6 - Communication failure due to socket timeout.

PHO_UNKNOWN_ERR := 99 - Unspecified internal error

PHO_PLANNING_FAILED = 201 - Trajectory planning has failed

PHO_NO_PART_FOUND = 202 - No part has been localized

PHO_NOT_INITIALIZED = 203 - Bin picking application has not been properly initialized on Vision Controller

PHO_PART_LOST = 204 - Part has been lost during motion execution

PHO_WRONG_BG_CONF := 255;;; - Wrong Binpicking configuration. For more details see Binpicking Studio console on Deployment Page


The following section provides a detailed explanation of two robot programs which must be adopted by the user before running binpicking for the first time - main_module.urp and calibration.urp.

4.4.1 main_module.urp

This is a basic bin picking template. The main program loop is defined here. The user is expected to reteach bin picking start and end positions, set IP Address and Port of Vision Controller and adopt placing part of the code to meet specific workcell and application requirements here.

Alt text
Figure 4.4.1

4.4.2 calibration.urp

NOTE: This feature is currently unavailable.

This procedure is used when you need to perform calibration from robot controller. Please note that recommended way of calibration is using Binpicking studio calibration wizard.

Reteach calibration positions and call pho_request_calib_add_point() requests after reaching each calibration pose. Users are recommended to run calibration in MANUAL mode in order to have full control over the process.

Always make sure that the robot does not collide with the workcell during transitions between specific waypoints.

Alt text

4.4.3 error_handling.urp

Several error situations might occur during bin picking procedures. Some problems are more serious, some less. For example, if no part is found or the trajectory planning fails, the program simply attempts to repeat the whole sequence. However, if a procedure call ends because of timeout, the program tries to reinitialize binpicking and start over. PHO_ERR_CODE contains the error state from the last response from the Vision Controller.

Alt text

4.5 Placing

For the purpose of orientated placing you can use information about picked object stored in following variables:

pho_tool_point_invariance - Tool invariance ID.

pho_gripping_point_id - Gripping point ID.

pho_gripping_point_invariance - Gripping point invariance ID.

4.6 Vision System ID

Some requests need an argument vision_system_id that defines selected vision system. Provided value must be the same as the ID of wanted vision system in Binpicking Studio. Studio starts indexing the vision systems from 1, but you can choose your preferred ID.

Alt text
Figure 4.1

In the following example we demonstrate basic usage of the main requests when using multiple vision systems.

NOTE: You can define different start/end pose for each vision system

  !send bin picking initialization request to the Vision Controller (first vision system
  pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget, 1;
  !send bin picking initialization request to the Vision Controller (second vision system
  pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget, 2;
  !call scan request to start localization for first vision system
  pho_request_scan 1;
  !wait for scan completition
  !call trajectory request and move to start pose
  pho_request_trajectory 1;
  MoveAbsJ start_bin_picking_jointtarget, v1000, z200, tool0;
  !receive trajectory and execute picking procedure

In next cycle you can call scan and trajectory requests for the second vision system or adapt program that better suits your needs.

5. Runtime

Check that the Bin Picking application is running on the Vision Controller side and is waiting for the connection:

Alt text
Figure 5.1

Load binpicking.urp program and click Play

Alt text
Figure 5.2

Move the manipulator to the Start Position. Select eitherAuto or Manual. When the Start Position is reached, confirm by clicking OK.

Alt text
Figure 5.3

After the Start Position has been confirmed click Play again. The Photoneo Module should now start running. If pho_wait_for_server() returns error, check the connection and network configuration.

Alt text
Figure 5.4

You should see a notification that a connection between UR robot controller and Vision Controller has been successfully established:

Alt text
Figure 5.5

At this point the robot should start sending requests to the Vision Controller and execute bin picking movements

NOTE: Ensure that you are ready to halt motion execution immediately. It is strongly recommended to reduce the speed to 10% of maximum during initial bin picking tests.