Bin Picking Tutorial: Setting up Photoneo Bin Picking Studio with ABB robots
Contents
1. Prerequisities
Photoneo ABB Interface is compatible with RobotWareOS versions 5.13 and higher.
In addition to compliant system version, following conditions must be met:
- (616-6) PC Interface Module available
- (672-1) Multitasking Module available
Browse to System Info -> System Properties -> Control Module -> Options to check if these options are available within your system:
NOTE: Use of Robot Studio v.6.06 (or higher version) is highly recommended for Photoneo ABB Interface setup.
2. ABB Controller setup
Following tutorial gives a step by step guide how to configure your ABB IRC5 controller and install all necessary RAPID files to get the robot interface for Photoneo Binpicking Solution running.
This tutorial was originally written using the latest RobotWare v.6.06, however it should be compatible with older versions with minor changes.
2.1 Network configuration
NOTE: Network configuration described below works only with RobotWareOS v.6 and higher. If you have an older version of RobotWareOS and need to change an IP address, please contact local ABB support.
Turn on the IRC5 Controller, wait for Initialization Screen to appear, open Menu screen and select Control Panel option:
On the Control Panel pane, select Controller Settings:
Select Settings and choose Network option:
Network settings window should appear. Use the touch panel keypad to set an IP address of the IRC5 WAN port:
Press OK button and restart the controller to apply new settings:
3 Robot Studio Setup
3.1 Connect to IRC5 Controller from Robot Studio
Set your local PC network settings to DHCP, plug Ethernet cable to Service Port of IRC5 Controller and launch Robot Studio.
For direct connection to Robot Controller, select Controller tab and One Click Connect option:
3.2 Transfer RAPID files
As a first step in installation of Photoneo ABB Interface, you need to copy RAPID files from Photoneo-ABB USB Stick you obtained within a delivery to the Robot Controller
You can use USB stick and copy files directly via pendant, however the most convenient way is to use the Robot Studio File Transfer tool.
Go to Controller tab and select File Transfer tool from the menu:
Photoneo ABB Interface consists of four RAPID files:
- MainModule.mod
- CustomerDefinitions.mod
- PhotoneoStateServer.mod
- PhotoneoCommon.sys
All these files need to be transferred to the HOME:/Photoneo/ folder created within the robot controller file system as shown by the figure below:
3.3 Create State Server Task
For correct functionality of State Server, open PhotoneoStateServer.mod and make sure that variables robot_task_id and pho_state_server_port are valid with respect to your system configuration.
(For example multi axes systems use T_ROB1 and T_ROB2 tasks, your network configuration might prevent using port 11005 etc). Keep default values if you are not confident about these changes.
As a next step we need to create State Server Task. Click Request Write Access button on the Controller tab in Robot Studio and Grant Access on the pendant:
State Server Task which runs in background, will report current robot state to the Vision Controller.
On the Left pane, click Controller then right click on the Task item in the list and select New Task...:
Following dialog should appear. Change settings as shown in the figure below:
(String to Copy & Paste: Task: PHO_STATE_SERVER, Main Entry: pho_state_server_main)
3.4 Automatic Loading of Modules
Now we need to configure ABB system to automatically load StateServer.mod during the boot.
Right Click on Automatic Loading of Modules and Select New Automatic Loading of Modules as shown in the figure below:
Configure module for PhotoneoState Server.mod as shown in the figure below:
(String to Copy & Paste: File: HOME://Photoneo/PhotoneoStateServer.mod)
Repeat the same procedure for PhotoneoCommon.sys as a system module:
(String to Copy & Paste: File: HOME://Photoneo/PhotoneoCommon.sys)
Repeat the same procedure also for CustomerDefinitions.mod containing user settings:
(String to Copy & Paste: File: HOME://Photoneo/CustomerDefinitions.mod)
3.5 Load Main Module
Now we are ready to manually load MainModule.mod to T_ROB1 task. You can load the MainModule.mod file from your PC or directly from the controller.
Right Click on T_ROB1 task and select Load Module or Load Module From Controller:
This was the last step in Photoneo ABB Interface installation. Your RAPID left panel should now look like this. Restart controller to apply all configured changes:
At this point your robot controller is configured to work with Photoneo Binpicking Solution. What is not ready is your RAPID code which must be adopted to meet your application requirements - reteach HOME, START and END poses, update part placing, application logic, etc. Following "RAPID" section provide basic examples how this can be programmed as well as detail explanation of Photoneo RAPID API.
4. RAPID
Photoneo ABB interface was designed to be easily integrated into existing application written in RAPID code. It provides two means which influence the final robot behavior:
- RAPID API - set of RAPID requests (or procedures) used to control the bin picking sequence. Requests are defined in PhotoneoCommon module and used within MainModule
- RAPID CONFIG - CustomerDefinitions.mod holds an application specific settings such as gripper commands, motion speed, precision and overall bin picking sequence definition.
4.1 RAPID API
Following API describes functionality of requests provided by Photoneo ABB Interface. They are intended for high level control of bin picking sequence and usually called in MainModule.mod.
pho_request_init(jointtarget pho_start_bin_picking_pose, jointtarget pho_end_bin_piking_pose) - request to initialize bin picking application on the Vision Controller side. Start and End Pose defined by the operator on the RAPID side are transferred to the Vision Controller and used in the trajectory planning pipeline as a start and terminus points.
pho_wait_for_server() - function to establish connection to the Vision Controller. Blocking function, program does not continue until connection is established properly. If server running on Vision Controller side is not running or not responding, en error is raised and RAPID program is terminated.
pho_request_scan() - request to trigger next scan and localization. Non-blocking request, RAPID program continues immediately at next line
pho_wait_for_scan_completition() - function to wait for scan to be finished. Should be called after triggering the scan, but not immediately
pho_request_trajectory() - request to start trajectory planning for current goal. Non blocking function, RAPID program continues immediately at the next line
pho_receive_trajectory() - function to receive requested bin picking trajectory (consists sequence of trajectory and gripper operations). This is usually called inside pho_bin_picking() function by default. Blocking function - waits for motion data to be received.
pho_request_scan_and_traj() - function to trigger scan, localization and trajectory planning in a single shot. Blocking function, handy for initial bin picking experiments. For actual production it is more convenient to split this sequence into particular actions and call them accordingly (pho_request_scan & pho_request_trajectory).
pho_request_calib_add_point() - request to add calibration point - scan is triggered and calibration is recalculated. Blocking request, program does not continue until calibration result is recalculated and received by the robot controller.
pho_calib_set() - request to set current calibration result to the PhoXi Scanner
pho_calib_reset() - request to reset current calibration and remove all previously added points
pho_bin_picking() - request to execute bin picking operation. It usually receives requested trajectory and perform motion execution. Blocking function, program does not continue until the whole sequence is finished. Always make sure that robot is in pho_start_bin_picking_pose before calling this function.
4.2 RAPID CONFIG
CustomerDefinitions module enables user to configure several bin picking settings, including gripper command implementations, trajectory speed and precision.
- open_gripper() - implement function for opening your gripper here
- close_gripper() - implement function for closing your gripper here
- gripper_user_1() - implement custom user gripper command (reserved for future use)
- gripper_user_2() - implement custom user gripper command (reserved for future use)
- gripper_user_3() - implement custom user gripper command (reserved for future use)
- gripper_user_4() - implement custom user gripper command (reserved for future use)
- gripper_user_5() - implement custom user gripper command (reserved for future use)
- gripper_user_6() - implement custom user gripper command (reserved for future use)
- binpicking_settings() - specify speed and precision settings for bin picking trajectories. Adopt number of trajectories to meet your bin picking setup. (by default 4 trajectories are used)
- pho_bin_picking_simple() - the very basic bin picking sequence (for experimental purposes)
- pho_bin_picking() - standard bin picking sequence. It is flexible, operation based implementation which allows performing bin picking sequences, which consist of various numbers of operations. Do not edit !!!
4.3 RAPID ERROR HANDLING
Photoneo BP solution provides basic error handling. If an error occurs during bin picking operation PHO_OCCURED_ERR flag is set to true. PHO_ERR_CODE variable contains code of specific error, which helps to identify the source of the problem.
PHO_NO_ERR := 0 - Service response from Vision Controller is valid
PHO_SERVICE_ERR := 1 - Service response from Vision Controller is invalid.
PHO_UNKNOWN_REQ := 2 - The Vision Controller received an unknown request.
PHO_COM_FAILURE := 3 - Communication failure due to socket closure.
PHO_BAD_DATA := 4 - Data validation check has failed.
PHO_TIMEOUT := 5 - Communication failure due to socket timeout.
PHO_PLANNING_FAILED := 201 - Trajectory planning has failed
PHO_NO_PART_FOUND := 202 - No part has been localized
PHO_NOT_INITIALIZED := 203 - Bin picking application has not been properly initialized on Vision Controller
PHO_PART_LOST := 204 - Part has been lost during motion execution
PHO_COLLISION_DETECT := 205 - Collision has been detected
PHO_UNKNOWN_ERR := 299 - Unspecified internal error
4.4 MAIN MODULE
Following section provides detail explanation of MainModule.mod - a module where user's application and Photoneo Bin Picking API are integrated together.
NOTE: If you use pre-generated program from Robot Studio simulation do not use MainModule.mod directly. In this case it serves only as an example how to use Photoneo RAPID API.
4.4.1 main()
The main() procedure is just a selection of which sub-procedure is about to be executed. Comment bin picking if you want to execute calibration and vice versa for bin picking execution.
MODULE MainModule ! Copyright (c) 2018 Photoneo s.r.o. ! All rights reserved ! Description: Main bin picking module ! ! Home, start and end binpicking target variables VAR robtarget home_pose; VAR robtarget start_bin_picking_robtarget; VAR robtarget end_bin_picking_robtarget; VAR jointtarget start_bin_picking_jointtarget; VAR jointtarget end_bin_picking_jointtarget; ! Error handling variables VAR num err_counter; CONST num MAX_ERR_COUNT := 2; PROC main() !calibration; ! If bin picking is about to be executed comment this line bin_picking; ! If calibration is about to be executed comment this line ENDPROC
4.4.2 bin_picking()
This is a basic bin picking template. The main program loop is defined here. User is expected to reteach bin picking start and end positions, set IP Address and Port of Vision Controller and adopt placing part of the code to meet specific workcell and application requirements here.
! BIN PICKING ! This is a basic bin picking template. The main program loop is defined here. User is expected to reteach ! bin picking start and end positions, set IP Address and Port of Vision Controller and adopt placing part ! of the code to meet specific workcell and application requirements here. ! PROC bin_picking() ! Clear error counter for err_handling procedure err_counter := 0; PHO_ERR_CODE := PHO_PLANNING_FAILED; ! ! RETEACH home position for your application home_pose := [[99999.0, 99999.0, 99999.0],[0, 0, 0, 1],[0,0,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]]; ! ! RETEACH bin picking start and end pose start_bin_picking_robtarget := [[99999.0, 99999.0, 99999.0],[0, 0, 0, 1],[0,0,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]]; end_bin_picking_robtarget := [[99999.0, 99999.0, 99999.0],[0, 0, 0, 1],[0,0,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]]; ! ! Convert cartesian position to jointtarget start_bin_picking_jointtarget := CalcJointT(start_bin_picking_robtarget, tool0); end_bin_picking_jointtarget := CalcJointT(end_bin_picking_robtarget, tool0); ! ! Connect to the the Vision Controller, adopt Vision Controller IP address and port here if needed pho_wait_for_server "192.168.1.6", 11004 ; ! ! Send bin picking initialization request to the Vision Controller, ! start and end poses defined above will be used in trajectory planning pipeline pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget; ! ! Move robot away from scanning area - reteach this position for your robot and workcell MoveJ home_pose, v500, z30, tool0; ! ! When robot is away from scanning area, trigger first scan and localization pho_request_scan; ! WHILE true DO scan_again: !==================== PHOTONEO BIN PICKING START =========================== ! Wait until scanning is completed pho_wait_for_scan_completion; ! ! Handle errors if occured IF (PHO_OCCURRED_ERR = TRUE) THEN err_handling; GOTO scan_again; ENDIF ! ! Trigger trajectory planning pho_request_trajectory; ! ! While trajectory is being calculated, move robot to bin picking start position MoveAbsJ start_bin_picking_jointtarget, v1000, z200, tool0; ! ! Execute bin picking application, calculated trajectory is received here pho_bin_picking; !==================== PHOTONEO BIN PICKING END =========================== ! ! Handle errors if occured IF (PHO_OCCURRED_ERR = TRUE) THEN err_handling; GOTO scan_again; ELSE ! Clear error counter for err_handling procedure err_counter := 0; ! !==================== PLACING START================================== ! Move robot from bin picking end position away from scanning area - reteach this position for your robot and workcell ! MoveJ [[171.40,-618.92,669.19],[9.66315E-05,-0.862481,-0.50609,-2.74185E-05],[-1,0,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v500, z30, tool0; ! ! Triger next scan and localization, trajectory for next cycle is calculated while object is being placed pho_request_scan; ! ! Commands for actual part placing - reteach positions for your robot and workcell ! MoveJ [[171.40,-618.92,669.19],[9.66315E-05,-0.862481,-0.50609,-2.74185E-05],[-1,0,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v500, z30, tool0; ! open_gripper; ! MoveJ [[171.40,-618.92,669.19],[9.66315E-05,-0.862481,-0.50609,-2.74185E-05],[-1,0,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v500, z30, tool0; !==================== PLACING END=================================== ENDIF ENDWHILE ENDPROC
4.4.3 calibration()
Reteach calibration positions and call pho_request_calib_add_point() request after reaching each calibration pose. It is recommended to run calibration in MANUAL mode step by step to have a full control over the process. Always make sure that robot does not collide with the workcell during transitions between specific waypoints.
! CALIBRATION ! Reteach calibration positions and call add calibration point request after reaching each calibration pose ! It it recomended to run calibration in MANUAL mode step by step to have a proper control over the process. ! Always make sure that robot does not collide with workcell during transitions between specific waypoints PROC calibration() ! Connect to the the Vision Controller, adopt Vision Controller IP address and port here if needed pho_wait_for_server "192.168.1.6", 11004; ! ! 1. calibration waypoint MoveJ [[726.48,-170.77,397.89],[0.100326,-0.586168,0.431152,-0.678565],[-1,0,-2,1],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v100, fine, tool0; pho_request_calib_add_point; ! ! 2. calibration waypoint MoveJ [[784.48,103.21,289.54],[0.015591,-0.569148,0.439403,-0.694804],[0,0,-2,1],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v100, fine, tool0; pho_request_calib_add_point; ! ! 3. calibration waypoint MoveJ [[612.57,314.22,592.92],[0.191405,-0.613209,0.508347,-0.573517],[0,0,-2,1],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v100, fine, tool0; pho_request_calib_add_point; ! ! 4. calibration waypoint MoveJ [[1007.35,-482.44,270.24],[0.566579,-0.514948,0.502784,0.40128],[-1,0,-2,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v100, fine, tool0; pho_request_calib_add_point; ! ! 5. calibration waypoint MoveJ [[975.17,-513.44,199.63],[0.615751,-0.53949,0.418442,0.393329],[-1,0,-2,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v100, fine, tool0; pho_request_calib_add_point; ! ! 6. calibration waypoint MoveJ [[813.58,-34.08,404.19],[0.317364,-0.830642,0.138416,-0.436068],[-1,1,0,0],[9E+09,9E+09,9E+09,9E+09,9E+09,9E+09]], v100, fine, tool0; pho_request_calib_add_point; ! ! Finally set calibration result to the PhoXi Scanner TPWrite "Setting calibration result to the PhoXi Scanner!"; pho_request_calib_set; TPWrite "Calibration complete!"; EXIT; ENDPROC
4.4.4 err_handling()
Several error situations might occur during bin picking procedure. Some of them are more serious some are less. For example if no part is found or trajectory planning fails, program just tries to repeat the whole sequence. However if communication failure is detected, program is halted immediately. PHO_ERR_CODE contains error state from last response from Vision Controller.
! ERROR HANDLING ! Shit happens. Several error situations might occur during bin picking procedure. Some of them are more serious some are less. ! For example if no part is found or trajectory planning fails, program just tries to repeat the whole sequence. However if ! communication failure is detected, program is halted immediately. It is possible to adopt this behavior here if needed PROC err_handling() !--Release gripper if needed !open_gripper; ! Move robot to home pose MoveJ home_pose, v500, z30, tool0; !--Display error TPWrite pho_err_info(PHO_ERR_CODE); IF err_counter >= MAX_ERR_COUNT THEN ErrLog 4800, "Photoneo internal error", pho_err_info(PHO_ERR_CODE), "Please reboot the Vision controller and restart application"," ", " "; EXIT; ENDIF ! If planning failed or no part was found, notify user and continue by next scan (Adopt if needed) IF (PHO_ERR_CODE = PHO_PLANNING_FAILED OR PHO_ERR_CODE = PHO_NO_PART_FOUND) THEN TPWrite "NO PART FOUND OR PLANNING PATH FAILED"; pho_request_scan; ! If bin picking has not been initialized or service returned error response, reinitialize and trigger new scan (Adopt if needed) ELSEIF(PHO_ERR_CODE = PHO_NOT_INITIALIZED) THEN pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget; pho_request_scan; ! If bin picking service returned error response, reinitialize and trigger new scan (Adopt if needed) ELSEIF (PHO_ERR_CODE = PHO_SERVICE_ERR ) THEN Waittime 10; pho_request_init start_bin_picking_jointtarget, end_bin_picking_jointtarget; pho_request_scan; ! In case of communication failure, rapid program is terminated immediatelly ELSEIF (PHO_ERR_CODE = PHO_BAD_DATA OR PHO_ERR_CODE = PHO_TIMEOUT OR PHO_ERR_CODE = PHO_COM_FAILURE ) THEN ErrLog 4800, "Photoneo internal error", "Communication failure!", "Please reboot the Vision controller"," "," "; EXIT; ! Otherwise trigger next scan and try to continue ELSE pho_request_scan; ENDIF err_counter := err_counter + 1; ! reset error flag for next cycle PHO_OCCURRED_ERR := FALSE; ENDPROC
5. Runtime
Apply RAPID changes, restart the robot controller and if everything is set correctly you should see following info screen on the pendant:
Make sure that Bin Picking application on Vision Controller side is running and waiting for the connection:
[[File: ]] (Tu asi pride screenshot z webu)
Open Main Menu, browse to Production Window and Set PP to Main - to the beginning of the main function:
Select if you want to run the application in AUTO or MANUAL mode and adopt the speed override if needed:
Hit the Play button in AUTO mode or Enable + Play button in MANUAL mode and you should see following application output:
You should see the notification that Vision Controller has established successful connection:
[[File: ]] (Tu asi pride screenshot z webu)
Robot should start sending requests to Vision Controller and execute bin picking movements now.