Difference between revisions of "Bin Picking Tutorial: Setting up Photoneo Bin Picking Studio with STAUBLI robots"

From 3D scanning Knowledge base - Photoneo wiki
Jump to: navigation, search
(3.4 Socket configuration)
(3.4 Socket configuration)
Line 133: Line 133:
  
 
'''Port:''' 11003 (Using different value is not recommended but can be done if needed. In this case please
 
'''Port:''' 11003 (Using different value is not recommended but can be done if needed. In this case please
contact support@photoneo.com to help you with configuring port in Binpicking Studio.)
+
contact '''support@photoneo.com''' to help you with configuring port in Binpicking Studio.)
  
 
'''Description:''' Photoneo Bin Picking Client
 
'''Description:''' Photoneo Bin Picking Client
Line 152: Line 152:
  
 
'''Name:''' PhotoneoStateServer
 
'''Name:''' PhotoneoStateServer
'''Port:''' 11004
+
 
 +
'''Port:''' 11004 (Using different value is not recommended but can be done if needed. In this case please
 +
contact '''support@photoneo.com''' to help you with configuring port in Binpicking Studio.)
 +
 
 
'''Description:''' Photoneo State Server
 
'''Description:''' Photoneo State Server
 +
 
'''Timeout:''' 0
 
'''Timeout:''' 0
 +
 
'''End of String:''' 10
 
'''End of String:''' 10
  

Revision as of 10:12, 7 May 2019

NOTE: Users are strongly recommended to read the general introduction to robot interfaces prior to installing specific robot modules.

1. Prerequisities

Photoneo Stäubli Interface was developed using the latest CS9 controler version s8.4.2Cs9BS1099.

It should be compatible with older CS8 controllers with several (mostly UI related) changes.

NOTE: Use of Stäubli Robotics Suite 2016.6.1 or higher is highly recommended for Photoneo Stäubli Interface setup.

2. CS9 Controller setup

The following tutorial provides a step by step guide of how to configure your Stäubli CS9 controller and install the VAL3 files you will need to get the robot interface for Photoneo Bin Picking Studio up and running.

2.1 Network configuration

The first step of the setup is to configure the network interface.

Turn on the CS9 Controller, wait for the Initialization Screen to appear, open Settings and select the Network option:


Alt text
Figure 2.1.1


As you can see in the figure below, two network interfaces are available.

- J204 is usually used for communication with third party devices, in this case with the Vision Controller.

- J205 is usually designated for file transfers between the Staubli Robotics Suite and the robot controller.

Input the J205 IP address to meet your network configuration.


Alt text
Figure 2.1.2

3 Staubli Robotics Suite

It is now time to launch Staubli Robotics Suite and start the SRS project configuration.

3.1 Create new project

Start by creating a new project. Click on the Home tab -> New -> New cell wizard.

Type the project name and select the project location. For the purposes of this tutorial we will use a TX2_40 arm and Photoneo_binpicking as a project name.


Alt text
Figure 3.1.1


If you are already connected to a real robot controller select Add a local controller from a remote controller.


Alt text
Figure 3.1.2


In order to select a controller from the available targets, click the "..." button


Alt text
Figure 3.1.3


In the General Information section type input the J205 IP address of the robot controller.

The Remote Connection section contains authorization information. Input the following credentials for accessing controller:

User Name: maintenance.

Password: spec_cal

Port : 5653

Hit OK and if everything works properly you should see the following dialog window:


Alt text
Figure 3.1.4


3.2 Copy VAL3 applications

If connection has been established properly, you are ready to copy the VAL3 files containing the Photoneo Staubli interface to your project.

The Photoneo Staubli interface consists of three folders - photoneo_common, customer_definitions and main_application.

Copy these folders from the Photoneo Staubli Module archive (.zip file) which you received with Photoneo Bin Picking Studio to the usrapp folder of your project (Project_name\Controller_name\usr\usrapp\ ).

Now open Cell Explorer in Staubli Robotics Suite and add the existing applications to the project.

Expand Project_name, right click on Controller_name, select Open Application and load all three applications (select .pjx files).


Alt text
Figure 3.2.1


The resulting project structure is shown in the figure below:


Alt text
Figure 3.2.2

3.3 Edit fBaseLink

The Cartesian origin of Staubli robots is different than the Cartesian origin of robot models used in the Photoneo Bin Picking Studio. In order to ensure successful calibration, an additional frame with a predefined offset in Z axis must be created.

In Cell Explorer, switch to the Data tab, under controller expand photoneo_common->frame and double click on fBaseLink. Edit the Z axis value according to the arm you are using for your application.


Alt text
Figure 3.3.1


A table of offsets for specific robot models is shown in the figure below:


Alt text
Figure 3.3.2

3.4 Socket configuration

In order to ensure proper communication with the Vision Controller, two sockets (one server and one client) need to be configured.

On the Home tab click on Physical IO's. In the IO table, right click on Sockets -> Edit Board


Alt text
Figure 3.4.1


First add Tcp_client which is used for transfering bin picking requests and responses between robot and the vision controller.


Alt text
Figure 3.4.2


Use the following values:

Name: PhotoneoClient

Port: 11003 (Using different value is not recommended but can be done if needed. In this case please contact support@photoneo.com to help you with configuring port in Binpicking Studio.)

Description: Photoneo Bin Picking Client

Timeout: 3000

End of String: 10

Server IP: IP address of Vision Controller

Click OK to apply the changes


Alt text
Figure 3.4.3


Now add Tcp_server with the following values:

Name: PhotoneoStateServer

Port: 11004 (Using different value is not recommended but can be done if needed. In this case please contact support@photoneo.com to help you with configuring port in Binpicking Studio.)

Description: Photoneo State Server

Timeout: 0

End of String: 10

Click OK to apply the changes


Alt text
Figure 3.4.4


We now need to create two sio variables and link them with existing sockets.

Using Cell Explorer switch to the Data tab and check two sio variables - sPHOCLIENT and sPHOSTATE.

In order to link sio variables with existing sockets, double click on sPHOCLIENT and select the Sockets\PhotoneoClient socket in the IO field.

Repeat the same procedure with sPHOSTATE and Sockets\PhotoneoStateServer.

You should now see that both variables are linked to physical IOs as is shown in the picture below:


Alt text
Figure 3.4.5


The initial steps of the SRS project configuration are now complete. Save the cell and you are ready to transfer the project files to the robot controller.

3.5 Transfer files

On the Home tab click on the Transfer Manager button and select target robot controller.


Alt text
Figure 3.5.1


If the connection between SRS and the robot controller has been set up properly, a transfer manager dialog window should now appear.

We have made changes to the IO and VAL3 Applications so confirm that these items are checked before transferring files.


Alt text
Figure 3.5.2


3.6 Load Application

If the project has been transferred to the robot controller successfully, it has to be loaded using the pendant. In the current version of the system, this is achieved using the "old menu" from the earlier versions of the controller.

Start by clicking the button shown in the picture below:


Alt text
Figure 3.6.1


In the old menu select Application manager -> Val3 applications -> main_application -> F6 (open)


Alt text
Figure 3.6.2


Return to the "color menu" by pressing the Home button. Now open the VAL3 Applications menu.

Click the bottom label and select the "V" letter on the widget as is shown in the picture below:


Alt text
Figure 3.6.3


You should see that main_application is now available in the application list.

As a final step, we need to set main_application to start automatically. Click main_application and set autostart mode.


Alt text
Figure 3.6.4


At this point your Robot Controller is configured to work with Photoneo Bin Picking Studio. However you should adapt the VAL3 code to meet the precise requirements of your application. This may involve reteaching HOME, START and END poses, updating part placing or application logic, etc. The following section of the tutorial provides some basic examples of how this can be programmed as well as a detailed explanation of the Photoneo VAL3 API.

4. VAL3

Photoneo Staubli interface was designed to be easily integrated into existing application written in VAL3 code. It provides two means which influence the final robot behavior:

- VAL3 API - set of VAL3 requests (or procedures) used to control the bin picking sequence. Requests are defined in photoneo_common application and used within main_module application

- VAL3 CONFIG - customer_definitions holds an application specific settings such as gripper commands, motion speed, precision and overall bin picking sequence definition.

4.1 VAL3 API

The following API describes the functionality of requests provided by Photoneo Staubli Interface. These requests are intended for high level control of bin picking sequences and are usually called in main_module application.


pho_request_init(joint x_jStartPose, joint x_jEndPose) - request to initialize bin picking application from the Vision Controller side. Start and End Poses defined by the operator side are transferred to the Vision Controller and used in the trajectory planning pipeline as start and terminus points.

wait_for_server() - optional function to check if a connection to the Vision Controller has been established.

scan_request() - request to trigger the next scan and localization. Non-blocking request; the VAL3 program continues immediately at the next line.

wait_for_scan_completition() - function to wait for scan to be finished. This should be called after triggering the scan, but not immediately

trajectory_request() - request to start trajectory planning for current goal. Non-blocking function; the VAL3 program continues immediately at the next line.

trajectory_receive() - function to receive requested bin picking trajectories (consisting of a sequence of trajectory and gripper operations). This is usually called inside pick_part() function by default. Blocking function; waits for motion data to be received.

calib_add_point_request() - request to add calibration point - a scan is triggered and the calibration is recalculated. Blocking request; the program will not continue until the calibration result is recalculated and has been received by the Robot Controller.

calib_set_to_scanner_request() - request to set current calibration result to the PhoXi Scanner

calib_reset_request() - request to reset current calibration and remove all previously added points

pick_part() - request to execute bin picking operation. This usually receives the requested trajectory and performs motion execution. Blocking function; the program does not continue until the whole sequence is finished.

bin_localization_request() - Only prepared for future

set_request_timeout(num x_nTimeout) - Set a timeout to a request. This procedure should be call before calling of request. If timeout is reached, then will set error variable photoneo_common:nErrCode as photoneo:nERR_TIMEOUT.


4.2 VAL3 CONFIG

customer_definitions enables the user to configure several bin picking settings, including gripper command implementations, trajectory speed and precision adjustments.


- gripper_attach() - implement function for attaching object to gripper here

- griper_detach() - implement function for detaching object to gripper here

- gripper_user_1() - implement custom user gripper command (reserved for future use)

- gripper_user_2() - implement custom user gripper command (reserved for future use)

- gripper_user_3() - implement custom user gripper command (reserved for future use)

- gripper_user_4() - implement custom user gripper command (reserved for future use)

- gripper_user_5() - implement custom user gripper command (reserved for future use)

- gripper_user_6() - implement custom user gripper command (reserved for future use)

- pick_part() - standard bin picking sequence. This flexible, operation-based implementation allows performing bin picking sequences, which consist of various numbers of operations to be performed. Do not edit !!!

- customer_definitions:mPickingSpeed which is type mdesc is used to define speed and path approximations for Picking Path Stages as defined in Photoneo Bin Picking Web Interface. Define trajectory speed and precision of Approach trajectory on the first line, for a Grasp trajectory on the second line and so on ... See the table below for better understanding:


Alt text
Figure 4.2.1



4.3 VAL3 ERROR HANDLING

Photoneo Bin Picking Studio provides basic error handling. If an error occurs during bin picking operations the photoneo_common:nErrCode variable contains the code of the specific error which will helps to identify the source of the problem.

photonoe_common:nERR_OK := 0 - Service response from Vision Controller is valid

photonoe_common:nERR_SERVICE := 1 - Service response from Vision Controller is invalid.

photonoe_common:nERR_UNKNOWN_REQ := 2 - The Vision Controller received an unknown request.

photonoe_common:nERR_COM_FAILURE := 3 - Communication failure due to socket closure.

photonoe_common:nERR_BAD_DATA := 4 - Data validation check has failed.

photonoe_common:nERR_TIMEOUT := 5 - Communication failure due to socket timeout.

photonoe_common:nERR_PLANNING_FAILED := 201 - Trajectory planning has failed

photonoe_common:nERR_NO_PART_FOUND := 202 - No part has been localized

photonoe_common:nERR_NOT_INITIALIZED := 203 - Bin picking application has not been properly initialized on Vision Controller

photonoe_common:nERR_PART_LOST := 204 - Part has been lost during motion execution

photonoe_common:nERR_UNKNOWN_ERR := 299 - Unspecified internal error


4.4 main_module application

The following section provides a detailed explanation of main_application - the module through which the user's application and Photoneo Bin Picking API are integrated together.

4.4.1 start()

The start() procedure is simply a selection of which particular task is about to be executed.

Comment PHOMAIN task in order to execute calibration procedure or comment PHOCALIB task if you want to execute bin picking.

   begin
     //----------------------------------------------------------------------
     // Copyright (c) 2018 Photoneo s.r.o.
     // All rights reserved
     // Description: Photoneo Staubli Module v.1.2.0 -  Start()
     //----------------------------------------------------------------------
     call photoneo_common:start()
     
     // Start PHOMAIN task
     // Comment this code, when you can calibrate
     if((taskStatus("PHOMAIN") == -1))
       taskCreate "PHOMAIN", 50, main()
     endIf      
     // Start PHOCALIB task 
     // Uncomment this code when you can calibrate
     //if((taskStatus("PHOCALIB") == -1))
     //  taskCreate "PHOCALIB", 50, calibration()
     //endIf
     // Start WATCHDOG task
     if((taskStatus("WATCHDOG") == -1))
       taskCreate "WATCHDOG", 50, watchdog_main()
     endIf           
   end

4.4.2 main()

This is a basic bin picking template. The main program loop is defined here. Adopt application logic and object placing to meet specific workcell and application requirements.

   begin
     //---------------------------------------------------------------
     // Copyright (c) 2018 Photoneo s.r.o.
     // All rights reserved
     // Description: Photoneo Staubli Module v.1.2.0 - Main Module
     //              Adopt this program to meet your requirements
     //---------------------------------------------------------------
     //                                         BIN PICKING
     // This is a basic bin picking template. The main program loop is defined here. User is expected to reteach 
     // bin picking start and end positions, set IP Address and Port of Vision Controller and adopt placing part
     // of the code to meet specific workcell and application requirements here. 
    
     // Clear error counter for err_handling procedure
     l_nErrorCounter = 0
     
     //  Wait for connection to the the Vision Controller 
     call photoneo_common:wait_for_server() 
     
     // Send bin picking initialization request to the Vision Controller, 
     call photoneo_common:initialize_request(jStartPose, jEndPose)
     
     // Move robot away from scanning area - reteach this position for your robot and workcell
     movej(jHomePose, flange, mNomSpeed)
     waitEndMove()
     
     // When robot is away from scanning area, trigger first scan and localization
     call photoneo_common:scan_request()
     
     while true
       //==================== PHOTONEO BIN PICKING START ===========================
       // wait until scanning is completed
       call photoneo_common:wait_for_scan_completition()
       
       // Check Error Status
       if(photoneo_common:nErrCode == photoneo_common:nOK)
       
         // Trigger trajectory planning
         call photoneo_common:trajectory_request()
       
         // While trajectory is being calculated, move robot to bin picking start position
         movej(jStartPose, flange, mNomSpeed)
         
         // If trajectory is valid pick part execute bin picking application
         call customer_definitions:pick_part()
         
       endIf 
        // Check Error Status
       if(photoneo_common:nErrCode == photoneo_common:nOK)
       
         //=================== PHOTONEO BIN PICKING END ===========================
       
         // Clear error counter if result is ok
         l_nErrorCounter = 0
         
         //==================== PLACING START==================================
         // Adopt code for placing operations
         movej(jHomePose, flange, mNomSpeed)
         waitEndMove()
         
         // Triger next scan and localization, trajectory for next cycle is calculated while object is being placed 
         call photoneo_common:scan_request()
       
         // ==================== PLACING START==================================
         // Implement place procedures
         // Print received info data about orientation grasped part. This data you can use for orientation placing
         call photoneo_common:printLog("Tool inv id: "+toString("1",photoneo_common:nToolInvID),"CLIENT")
         call photoneo_common:printLog("Grip id: "+toString("1",photoneo_common:nGrippingPointID),"CLIENT") 
         call photoneo_common:printLog("Grip inv id: "+toString("1",photoneo_common:nGrippingPointInvID),"CLIENT")
         //=====================================================================
         
       else
         
         //                                         ERROR HANDLING
         // Shit happens. Several error situations might occur during bin picking procedure. Some of them are more serious some are less.
         // For example if no part is found or trajectory planning fails, program just tries to repeat the whole sequence. However if 
         // communication failure is detected, program is halted immediately. It is possible to adopt this behavior here if needed
         
         // In case of communication failure, VAL3 program is terminated immediatelly
         if(photoneo_common:nErrCode == photoneo_common:nERR_COM_FAILURE or photoneo_common:nErrCode == photoneo_common:nERR_BAD_DATA or photoneo_common:nErrCode == photoneo_common:nERR_TIMEOUT)
           popUpMsg("Communication failure")
           movej(jHomePose, flange, mNomSpeed)
           waitEndMove()
           return
           
           //If bin picking service returned error response, reinitialize and trigger new scan (Adopt if needed) 
         elseIf(photoneo_common:nErrCode == photoneo_common:nERR_SERVICE or photoneo_common:nErrCode == photoneo_common:nERR_NOT_INITIALIZED)
           popUpMsg("Service error from VC")
           movej(jHomePose, flange, mNomSpeed)
           waitEndMove()
           return
       
           //  If planning failed or no part was found, notify user and continue by next scan (Adopt if needed)
         elseIf(photoneo_common:nErrCode == photoneo_common:nERR_PLANNING_FAILED or photoneo_common:nErrCode == photoneo_common:nERR_NO_PART_FOUND)
           popUpMsg("No part found or planning failed")
           movej(jHomePose, flange, mNomSpeed)
           waitEndMove()
           call photoneo_common:scan_request()
         endIf
         
         l_nErrorCounter = l_nErrorCounter + 1
         if(l_nErrorCounter > 2)
           return
         endIf
         
       endIf   
     endWhile
   end

4.4.3 calibration()

This procedure is useful only if you need make a calibration from robot controller. Better way is used Binpicking studio for calibration.

In main_application in procedure start() comment PHOMAIN task in order to execute calibration procedure and uncomment PHOCALIB. If you will finish, then return back changes

Reteach calibration positions and call pho_request_calib_add_point() requests after reaching each calibration pose. Users are recommended to run calibration in MANUAL mode in order to have full control over the process.

Always make sure that the robot does not collide with the workcell during transitions between specific waypoints.

   begin
     
     //----------------------------------------------------------------------
     // Copyright (c) 2018 Photoneo s.r.o.
     // All rights reserved
     // Description: Photoneo Staubli Module v.1.2.0 -  Calibration Routine
     //              Reteach points and use as many as you need
     //----------------------------------------------------------------------
     
     //                                         CALIBRATION
     // Reteach calibration positions and call add calibration point request after reaching each calibration pose
     // It it recomended to run calibration in MANUAL mode step by step to have a proper control over the process. 
     // Always make sure that robot does not collide with workcell during transitions between specific waypoints
     
     //  Wait for connection to the the Vision Controller 
     call photoneo_common:wait_for_server() 
     
     // Send request to starting calibration
     call photoneo_common:calib_start_request()
     
     movej(jCalib1, flange, mNomSpeed)
     waitEndMove()
     call photoneo_common:printLog("Calibration Point 1 reached", "CLIENT")
     call photoneo_common:calib_add_point_request()
      
     movej(jCalib2, flange, mNomSpeed)
     waitEndMove()
     call photoneo_common:printLog("Calibration Point 2 reached", "CLIENT")
     call photoneo_common:calib_add_point_request()
        
     movej(jCalib3, flange, mNomSpeed)
     waitEndMove()
     call photoneo_common:printLog("Calibration Point 3 reached", "CLIENT")
     call photoneo_common:calib_add_point_request()
         
     movej(jCalib4, flange, mNomSpeed)
     waitEndMove()
     call photoneo_common:printLog("Calibration Point 4 reached", "CLIENT")
     call photoneo_common:calib_add_point_request()
     
     movej(jCalibReserved1, flange, mNomSpeed)
     waitEndMove()
     call photoneo_common:printLog("Calibration Helping WayPoint reached", "CLIENT")
     
     movej(jCalib5, flange, mNomSpeed)
     waitEndMove()
     call photoneo_common:printLog("Calibration Point 4 reached", "CLIENT")
     call photoneo_common:calib_add_point_request()
     
     call photoneo_common:calib_set_to_scanner_request()
      
   end

Explore the calibration request:

calib_start_request() - request to start calibration

calib_add_point_request() - request to add calibration point - a scan is triggered and the calibration is recalculated. Blocking request; the program will not continue until the calibration result is recalculated and has been received by the Robot Controller.

calib_set_to_scanner_request() - request to set current calibration result to the PhoXi Scanner

calib_reset_request() - request to reset current calibration and remove all previously added points

4.4.5 watchdog_main() and error handling

Several error situations might occur during bin picking procedures. Some problems are more serious, some less. For example, if no part is found or the trajectory planning fails, the program simply attempts to repeat the whole sequence. However, if a communication failure is detected, the program is halted immediately. photoneo_common:nErrCode contains the error state from the last response from the Vision Controller. Example of Error handling is described in chapter 4.4.2

IMPORTANT: If error occurs during bin picking procedure, robot moves to home position and halt PHOMAIN task. Watchdog monitors state of PHOMAIN() every second and restart this task imediately.

   begin
     //------------------------------------------------------------------------------------------------------------------------
     // Copyright (c) 2018 Photoneo s.r.o.
     // All rights reserved
     // Description: Photoneo Staubli Module v.1.2.0 -  Watchdog
     //              Monitor PHOMAIN task status and restart task after crash
     //------------------------------------------------------------------------------------------------------------------------
     
     while(true)
       // Check PHOMAIN State
       if((taskStatus("PHOMAIN") == -1))
           popUpMsg("Main program is down! Restarting...")
         taskCreate "PHOMAIN",   50, main()
       endIf
       // Check Tasks State Every Second
       delay(1)
     endWhile      
   end

4.5 Placing

For orientation placing you can use information about grasped object. The informations are included in variables:

photoneo_common:nToolInvID - Tool invariance ID.

photoneo_common:nGrippingPointID - Gripping point ID.

photoneo_common:nGrippingPointInvID - Gripping point invariance ID.

4.6 Multiple vision systems

If you are using multiple vision system from Binpicking studio, then must be define vision system, which will be use in request. A vision system is defined in variable photoneo_common:nVisionSystemID. Before calling a request must be set vision system ID to the variable. Short example how to send initialize request to 2 vision system:

     // Send bin picking initialization request to the Vision Controller (First vision system)
     photoneo_common:nVisionSystemID = 0
     call photoneo_common:initialize_request(jStartPose, jEndPose)
     
     // Send bin picking initialization request to the Vision Controller (Second vision system)
     photoneo_common:nVisionSystemID = 1
     call photoneo_common:initialize_request(jStartPose, jEndPose)

5. Runtime

Now you are ready to start the bin picking application. Save the Cell, transfer all changes to the robot controller using Transfer Manager tool, and reload the application using the "old menu".

Make sure that the Bin Picking application on the Vision Controller side is running and waiting for a connection.

Switch controller to Auto mode, press Reset button if Safety Restart is needed and launch the main_application.

If a connection to Vision Controller was established properly you should see following information on logging screen:


Alt text
Figure 5.1


You should also receive the notification that the Vision Controller has established a successful connection to the robot controller:


Alt text
Figure 5.2


According to the current robot controller mode you might need to press the Power (Button I) and the Play (Button II) in order to enable motion execution


Alt text
Figure 5.3


Robot controller should start sending requests to the Vision controller and executing first bin picking movements. See logging screen on Pendant for information about the procedure:


Alt text
Figure 5.4


NOTE: Ensure that you are ready to halt motion execution immediately. It is strongly recommended to reduce the speed to 10% of maximum during initial bin picking tests.