Virtual Brain Lab

About the VBL

  • Overview
  • Team
  • Join Us

Pinpoint

  • Installation and Use
  • Allen CCF to In Vivo Alignment
  • Electrophysiology Atlas
  • Sensapex Link
    • Installation
      • For usage like a standalone app/server
      • For usage like a library
      • To develop this package
    • Usage
      • Registering a manipulator
        • Example
      • Calibrating a manipulator
        • Example
      • Bypassing calibration
        • Example
      • Enable movement
        • Example
      • Get a manipulator’s position
      • Set position of a manipulator
      • Drive to depth
        • Example
      • Set inside brain
        • Example
      • Emergency Stop
        • Example
    • Code Organization
      • server.py
      • sensapex_handler.py
      • manipulator.py
    • General code practices (for developers looking to contribute)
  • Development

Urchin

  • Installation and Use
  • Development

Session Viewer

  • Session Viewer
  • IBL Average Trial

Misc

  • VBL Core
  • Addressables Storage
  • General Programming Advice
  • Unity

API Reference

  • API Reference
Virtual Brain Lab
  • »
  • Sensapex Link
  • View page source

Sensapex Link

The Sensapex Link is a python WebSocket server that allows any WebSocket compliant application (such as Pinpoint (Neuropixels Trajectory Planner)) to have limited communication with Sensapex uMp Micromanipulators

Table of Contents

  • Installation

  • Usage

  • Code Organization

Installation

Copied from the repo’s README.md

For usage like a standalone app/server

  1. Ensure Python 3.8+ and pip are installed

  2. pip install nptraj-sensapex-link

  3. Run python -m nptraj-sensapex-link to start the server

For usage like a library

  1. Ensure Python 3.8+ and pip are installed

  2. pip install nptraj-sensapex-link

  3. Use from nptraj_sensapex_link import launch and call launch() to start the server

    1. Alternatively, use import nptraj_sensapex_link and call nptraj_sensapex_link.launch()

To develop this package

  1. Ensure Python 3.8+ and pip are installed

  2. Clone the repo

  3. cd nptraj-sensapex-link and run pip install -r requirements.txt

  4. The package is located in src/

  5. python src/nptraj_sensapex_link/server.py launches the server

  6. Unit tests are available to run under the tests/ directory

Usage

This is a list of available WebSocket events. The code shown is pseudo-WebSocket code that can be used to interact with the server. The exact implementation will depend on the platform and WebSocket interface used.

In general:

  • Each event will take in an input and call a callback function with certain arguments

  • Before a manipulator can be used, it must be registered and calibrated

    • Before a manipulator can be moved, it must have its movement enabled

    • A manpulator’s position can be read before its movement is enabled though

  • The server will log unknown events, but will not return callback arguments or emit any messages

Table of Contents

  • Registering a manipulator

  • Calibrating a manipulator

    • Bypassing calibration

  • Get a manipulator’s position

  • Set position of a manipulator

  • Drive manipualtor to depth

  • Set “inside brain” state of a manipulator

  • Enable movement

  • Emergency Stop

Registering a manipulator

Every manipulator in a Sensapex setup must be registered to the server before being used.

Event: register_manipulator

Expected Arguments:

  • Manipulator ID: int

Callback Responses (int, string):

  • (manipulator_id, ''): No errors, registered manipulator with ID manipulator_id

  • (manipulator_id, 'Manipulator already registered'): Manipulator is already registered, no action taken

  • (manipulator_id, 'Manipulator not found'): The manipulator is not discoverable by the API and may be disconnected or offline

  • (manipulator_id, 'Error registering manipulator'): An unknown error has occurred while registering

Example

# Register manipulator with ID 1
ws.emit('register_manipulator', 1, callback=my_callback_func)

Calibrating a manipulator

To ensure all manipulators are working properly before applying autonomous control, all manipulators must have their movement checked and calibrated. This is done by moving all four axes through their full range of motion while also invoking the calibrate functionality.

Event: calibrate

Expected Arguments:

  • Manipulator ID: int

Callback Responses (int, string):

  • (manipulator_id, ''): No errors, calibrated manipulator with ID manipulator_id

  • (manipulator_id, 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, 'Error calling calibrate'): A Sensapex SDK error has occurred while calibrating

  • (manipulator_id, 'Error calibrating manipulator'): An unknown error has occurred while calibrating

Example

# Calibrate manipulator 1
ws.emit('calibrate', 1, callback=my_callback_func)

Bypassing calibration

FOR TESTING PURPOSES ONLY!! Do not use in production code.

The calibration requirement may be bypassed by sending this event.

Event: bypass_calibration

Expected Arguments:

  • Manipulator ID: int

Callback Responses (int, string):

  • (manipulator_id, ''): No errors, calibration bypassed for manipulator with ID manipulator_id

  • (manipulator_id, 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, (), 'Manipulator not calibrated'): Manipulator is not calibrated yet

  • (manipulator_id, 'Error bypassing calibration'): An unknown error has occurred while bypassing calibration

Example

# Bypass calibration for manipulator 1
ws.emit('bypass_calibration', 1, callback=my_callback_func)

Enable movement

To prevent accidental movement commands, a manipulator must have its movement feature enabled. A manipulator may have its movement enabled for set period of time or enabled indefinitely. Relevant information is passed through the event. Once a write lease has expired, an event is emitted back to the server with the ID of the manipulator which can no longer write as the payload.

Event: set_can_write

Expected Arguments (dictionary/object with the following format):

  • manipulator_id: int

  • can_write: bool

  • hours: float

Callback Responses (int, bool, string)

  • (manipulator_id, can_write, ''): No errors, set state is returned

  • (manipulator_id or -1, False, 'Invalid data format'): Invalid/unexpected argument format

  • (manipulator_id or -1, False, 'Error in set_can_write'): An unknown error occurred while starting this function

  • (manipulator_id, False, 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, False, 'Manipulator not calibrated'): Manipulator is not calibrated yet

  • (manipulator_id, False, 'Error setting can_write'): An unknown error has occurred enabling movement

Reponse Event: write_disabled

Payload: manipulator_id: int

Example

```python
# Enable movement for manipulator 1 indefinitely (0 = indefinite hours)
ws.emit('set_can_write', {
    'manipulator_id': 1,
    'can_write': True,
    'hours': 0
})

Get a manipulator’s position

Receive the position of a specified manipulator as X, Y, Z, W (depth) in µm from the origin.

Event: get_pos

Expected Arguments:

  • Manipulator ID: int

Callback Responses (int, float[4], string)

  • (manipulator_id, (x, y, z, w), ''): No errors, position is returned

  • (manipulator_id, (), 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, (), 'Manipulator not calibrated'): Manipulator is not calibrated yet

  • (manipulator_id, (), 'Error getting position'): An unknown error has occured while getting position

# Gets the position of manipulator 1
ws.emit('get_pos', 1, callback=my_callback_func)

Set position of a manipulator

Instructs a manipulator to go to a position relative to the origin in µm.

Manipulators move asynchronously from each other. This means large batches of movement events can be sent to the server for several manipulators and each manipulator will move through the events assigned to them independently.

When a manipulator is set to be “inside” the brain, it will have all axes except the depth axis locked. This is to prevent accidental lateral movement while inside brain tissue. This state is set by set_inside_brain. One may also explicitly specify movement in only the depth axis using drive_to_depth

Event: goto_pos

Expected Arguments (dictionary/object with the following format):

  • manipulator_id: int

  • pos: float[4] (in x, y, z, w as µm from the origin)

  • speed: int (in µm/s)

Callback Responses (int, float[4], string)

  • (manipulator_id, (x, y, z, w), ''): No errors, final position is returned

  • (manipulator_id or -1, (), 'Invalid data format'): Invalid/unexpected argument format

  • (manipulator_id or -1, (), 'Error in goto_pos'): An unknown error occured while starting this function

  • (manipualtor_id, (), 'Manipulator movement canceled'): Emergency stop was used and manipulator movements have been canceled

  • (manipulator_id, (), 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, (), 'Manipulator not calibrated'): Manipulator is not calibrated yet

  • (manipulator_id, (), 'Error moving manipulator'): An unknown error has occured while getting position

# Set manipulator 1 to position 0, 0, 0, 0 at 2000 µm/s
ws.emit('goto_pos', {
    'manipulator_id': 1,
    'pos': [0, 0, 0, 0],
    'speed': 2000
})

Drive to depth

Instructs a manipulator to go to a specific depth in µm. This is equivalent to setting the position of the manipulator to the same position but with a different depth. This function helps to explicitly make sure no other axis except the depth axis is moving during a movement call.

Event: drive_to_depth

Expected Arguments (dictionary/object with the following format):

  • manipulator_id: int

  • depth: float (in µm from the origin)

  • speed: int (in µm/s)

Callback Responses (int, float, string)

  • (manipulator_id, depth, ''): No errors, final position is returned

  • (manipulator_id or -1, 0, 'Invalid data format'): Invalid/unexpected argument format

  • (manipulator_id or -1, 0, 'Error in drive_to_depth'): An unknown error occured while starting this function

  • (manipulator_id, 0, 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, 0, 'Manipulator not calibrated'): Manipulator is not calibrated yet

  • (manipulator_id, 0, 'Error driving manipulator'): An unknown error has occurred while driving to depth

Example

# Drive manipulator 1 to a depth of 1000 µm at 2000 µm/s
ws.emit('drive_to_depth', {
    'manipulator_id': 1,
    'depth': 1000,
    'speed': 2000
})

Set inside brain

Sets the “inside brain” state of a manipulator. When a manipulator is inside the brain, it will have all axes except the depth axis locked. This is to prevent accidental lateral movement while inside brain tissue.

Event: set_inside_brain

Expected Arguments (dictionary/object with the following format):

  • manipulator_id: int

  • inside: bool

Callback Responses (int, bool, string)

  • (manipulator_id, inside, ''): No errors, set state is returned

  • (manipulator_id or -1, False, 'Invalid data format'): Invalid/unexpected argument format

  • (manipulator_id or -1, False, 'Error in set_inside_brain'): An unknown error occurred while starting this function

  • (manipulator_id, False, 'Manipulator not registered'): Manipulator is not registered yet

  • (manipulator_id, False, 'Manipulator not calibrated'): Manipulator is not calibrated yet

  • (manipulator_id, False, 'Error setting inside brain'): An unknown error has occurred while setting inside brain

Example

# Set manipulator 1 to be inside the brain
ws.emit('inside_brain', {
    'manipulator_id': 1,
    'inside': True
})

Emergency Stop

There are two ways an emergency stop can be triggered: through this event or the hardware/serial attached button. The server will connect to the first serial device it finds which names itself “USB Serial Device” (which is what an Arduinos would appear as) and listen for any serial input from this source (at a baud rate of 9600). The system will poll the serial port every 50 ms to check.

Both the WebSocket event and the serial method will stop all movement, remove all movement in the queue, and set all manipulators to be uncalibrated. Therefore, one must recalibrate the manipulators before continuing.

Event: stop

Expected Arguments (dictionary/object with the following format):

  • None

Callback Responses (int, bool, string)

  • true: No errors, all movement stopped

  • false: An unknown error has occurred while stopping all movement

Example

# Stop all movement
ws.emit('stop')

Code Organization

There are three main components of this package:

  1. server.py: The main server code

  2. sensapex_handler.py: Code for communicating with the Sensapex API

  3. manipulator.py: A class for representing a manipulator

server.py

All code responsible for WebSocket server functionality is in this file. This includes client connection/disconnection, event handling, and the main server loop.

For every event, the server does the following:

  1. Extract the arguments passed in the event

  2. Log that the event was received

  3. Call the appropriate function in sensapex_handler.py with the arguments

  4. Call the callback function with the response from sensapex_handler.py as parameters

sensapex_handler.py

All code responsible for communicating with the Sensapex API is in this file. This loading the DLL, establishing a connection with the equipment and maintaining a dictionary of registered manipulators.

Functions names here are the same as the WebSocket events. They are called when the server receives an event from a client. In general, the function does the following:

  1. Recieve extracted arguments from server.py

  2. Inside try/except block, call the appropriate Sensapex API function

  3. Log/handle successes and failures

  4. Return the callback parameters to server.py

manipulator.py

To help make calling functions on manipulators easier, a custom class containing the necessary functions and flags is created. This class is used by sensapex_handler.py to call manipulator-specific APIs such as get_pos (get position) and to keep track of manipulator-specific flags such as if it has been calibrated yet.

Manipulator functions handle errors and return the appropriate callback parameters like in sensapex_handler.py.

General code practices (for developers looking to contribute)

  • Type hinting is implemented where possible

  • Tuples are used when possible

  • Only one client can be connected at a time

Previous Next

© Copyright 2022, Daniel Birman.

Built with Sphinx using a theme provided by Read the Docs.