nicoface

The nicoface package contains classes to control the facial expressions of the NICO robot.

faceExpression

The faceExpression class allows manipulation and generation of the facial expressions of the NICO robot

class nicoface.FaceExpression.faceExpression(devicename=None, simulation=False)

Bases: object

The FaceExpression class provides an interface to manipulate NICO’s facial expressions

_calculate_transition(start, target, steps=3, padding=False)

Interpolates between start and target values.

Parameters

  • start (list) – initial list

  • target (list) – target list

  • steps (int) – Number of interpolation steps

  • padding (whether lists with missmatched sizes should be zero padded) – Number of interpolation steps

_scan_ports(baudrate, timeout)

Automatically detects and establishes a connection with the FaceExpression Arduino

_send(message, expected_response)

Sends message to serial device and verifies if expected_response was returned

Parameters

  • message (str) – message to send

  • expected_response (str) – response the serial device is expected to return

draw_wavelet(length, ystr1, yoff1, xstr1, xoff1, image)

Generates a ricker wavelet for given parameters and draws them into the given PIL image.

Parameters

  • length (float) – timeframe

  • dt (float) – samplerate

  • xoff (float) – offset in x position

  • xstr (float) – stretch in x position

  • yoff (float) – offset y position

  • ystr (float) – stretch in y position

  • image (PIL image) – image in which to draw the line

Returns

modified image

Return type

PIL image

gen_eyebrowse(ml1=(0.1, 0.4, 1, - 0.55), file_name=None, type='l')

Generation of the specified eyebrow. The eyebrow is specified by setting type to “l” or “r” for left or right eyebrow.

Parameters

  • ml1 (tuple(float,float,float,float)) – line of the eyebrow (stretch_in_y_position, offset_y_position, stretch_in_x_position , offset_x_position)

  • file_name (str) – name of the file to save the resulting plot to (if desired)

  • type (str) – specifies which eyebrow to generate (“l”, “r”)

gen_mouth(ml1=(- 1.4, 0.7, 1.0, 0), ml2=(None, None, None, None), file_name=None)

Generation of the mouth. One or two lines can be generated.

Parameters

  • ml1 (tuple(float,float,float,float)) – first line of the mouth (stretch_in_y_position, offset_y_position, stretch_in_x_position , offset_x_position)

  • ml2 (tuple) – second line of the mouth (stretch_in_y_position, offset_y_position, stretch_in_x_position , offset_x_position)

  • file_name (str) – name of the file to save the resulting plot to (if desired)

Returns

PIL image of the mouth

Return type

PIL.Image

generate_polynomial_eyebrow(degs, x_shift=0, crop_left=0, crop_right=0, left=True)

Generates eyebrow curve using an n-degree polynomial y = sum(degs[i] * x ** i). The origin point can be shifted with x_shift, the crop_left and crop_right parameters allow creating black borders at either side of the image to reduce size of the eyebrow.

Parameters

  • degs (list) – factor for each degree of the polynomial

  • x_shift (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • crop_left (int) – black border in pixels on the left of the resulting image

  • crop_right (int) – black border in pixels on the right of the resulting image

  • left (bool) – whether left or right eyebrow is generated

generate_polynomial_mouth(degs1, degs2, x_shift=7.45, crop_left=0, crop_right=0)

Generates mouth curves using two n-degree polynomial y = sum(degs[i] * x ** i). The origin point can be shifted with x_shift, the crop_left and crop_right parameters allow creating black borders at either side of the image to reduce size of the mouth.

Parameters

  • degs1 (list) – factor for each degree of the first polynomial mouth line

  • degs2 (list) – factor for each degree of the second polynomial mouth line

  • x_shift (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • crop_left (int) – black border in pixels on the left of the resulting image

  • crop_right (int) – black border in pixels on the right of the resulting image

morph_face_expression(target_preset, steps=3, delay=0.0)

Morphs displayed face to given preset. This requires the displayed face expression to be fully polynomial or fully wavelet based.

Known presets: ‘happiness’, ‘sadness’, ‘anger’, ‘disgust’, ‘surprise’,

‘fear’, ‘neutral’

Parameters

  • target_preset (str) – name of the target expression (‘happiness’, ‘sadness’, ‘anger’, ‘disgust’, ‘surprise’, ‘fear’, ‘neutral’)

  • steps (int) – number of transition steps

  • delay (float) – delay between transition steps

morph_polynomial_face(m1_target, m2_target, m_x_shift_target, m_crop_left_target, m_crop_right_target, left_target, l_x_shift_target, l_crop_left_target, l_crop_right_target, right_target, r_x_shift_target, r_crop_left_target, r_crop_right_target, steps=3, delay=0.0)

Morphs displayed face into the given target face in the given number of transition steps. This requires all face images to be polynomial based.

Parameters

  • m1_target (list) – factor for each degree of the first polynomial mouth line

  • m2_target (list) – factor for each degree of the second polynomial mouth line

  • m_x_shift_target (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • m_crop_left_target (int) – black border in pixels on the left of the resulting mouth image

  • m_crop_right_target (int) – black border in pixels on the right of the resulting mouth image

  • left_target (list) – factor for each degree of the left eyebrow polynomial

  • l_x_shift_target (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • l_crop_left_target (int) – black border in pixels on the left of the resulting eyebrow image

  • l_crop_right_target (int) – black border in pixels on the right of the resulting eyebrow image

  • right_target (list) – factor for each degree of the right eyebrow polynomial

  • r_x_shift_target (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • r_crop_left_target (int) – black border in pixels on the left of the resulting eyebrow image

  • r_crop_right_target (int) – black border in pixels on the right of the resulting eyebrow image

  • steps (int) – number of transition steps

  • delay (float) – delay between transition steps

morph_wavelet_face(m1_target, m2_target, left_target, right_target, steps=3, delay=0.0)

Morphs displayed face into the given target face in the given number of transition steps. This requires all face images to be wavelet based.

Parameters

  • m1_target (tuple(float, float, float, float)) – wavelet parameters for first mouth line (stretch_in_y_position, offset_y_position, stretch_in_x_position , offset_x_position)

  • m2_target (tuple(float, float, float, float)) – wavelet parameters for second mouth line

  • left_target (tuple(float, float, float, float)) – wavelet parameters for left eyebrow

  • right_target (tuple(float, float, float, float)) – wavelet paramters for right eyebrow

  • steps (int) – number of transition steps

  • delay (float) – delay between transition steps

np_to_str(pix)

Packs a numpy array into a String for serialization

Parameters

pix (numpy.array) – array to pack

Returns

Numpy array packed into a string

Return type

str

polynomial(x, degs)

Calculates y values for given x values using an n-degree polynomial: y = sum(degs[i] * x ** i)

Parameters

  • x (list) – x values to calculate y for

  • degs (list) – factor for each degree of the polynomial

polynomial_presets = {'anger': {'left': [[2.5, 0.7, 0, 0, 0], 0, 0, 0], 'mouth': [[2, 0, 0.005, 0, 0.002], [8, 0, -0.005, 0, -0.002], 7.45, 2, 2], 'right': [[7.4, -0.7, 0, 0, 0], 0, 0, 0]}, 'disgust': {'left': [[8.75, -0.25, 0, 0, 0], 0, 0, 0], 'mouth': [[4, 0, -0.39, 0, 0.0385], [7, 0, 0.39, 0, -0.0385], 7.45, 4, 4], 'right': [[7, 0.25, 0, 0, 0], 0, 0, 0]}, 'fear': {'left': [[3, -0.25, 0, 0, 0], 0, 0, 0], 'mouth': [[3, 0, -0.005, 0.0, 0.00075], [7, 0, 0.005, 0.0, -0.00075], 7.45, 1, 1], 'right': [[1.25, 0.25, 0, 0, 0], 0, 0, 0]}, 'happiness': {'left': [[4.75, -0.25, 0, 0, 0], 0, 0, 0], 'mouth': [[7, 0, -0.11, 0, 0], [7, 0, -0.11, 0, 0], 7.45, 2, 2], 'right': [[3, 0.25, 0, 0, 0], 0, 0, 0]}, 'neutral': {'left': [[4, 0, 0, 0, 0], 0, 0, 0], 'mouth': [[5, 0, 0, 0, 0], [5, 0, 0, 0, 0], 7.45, 2, 2], 'right': [[4, 0, 0, 0, 0], 0, 0, 0]}, 'sadness': {'left': [[5, -0.4, 0, 0, 0], 0, 0, 0], 'mouth': [[5, 0, 0.075, 0, 0], [5, 0, 0.075, 0, 0], 7.45, 2, 2], 'right': [[2.2, 0.4, 0, 0, 0], 0, 0, 0]}, 'surprise': {'left': [[5, -1, 0.125, 0, 0], 0, 0, 0], 'mouth': [[2, 0, 0.005, 0, 0.0045], [8, 0, -0.005, 0, -0.0045], 7.45, 3, 3], 'right': [[4, -0.5, 0.095, 0, 0], 0, 0, 0]}}
ricker(f, length=0.512, dt=0.001, xoff=0, xstr=1, yoff=0, ystr=1)

Generates a Ricker wavelet (based on https://agilescientific.com/blog/2013/12/10/to-plot-a-wavelet.html)

Parameters

  • f (float) – peak frequency

  • length (float) – timeframe

  • dt (float) – samplerate

  • xoff (float) – offset in x position

  • xstr (float) – stretch in x position

  • yoff (float) – offset y position

  • ystr (float) – stretch in y position

Returns

Tuple: time_axis, function_values

Return type

tuple

send(address='all')

Send current expression (“all”) or a specific part of it to display. (“m” for mouth, “l” or “r” for left or right eyebrow)

Parameters

address (str) – part of face to send (all, l, r, m)

sendFaceExpression(expression)

Changes NICO’s facial expression to the given preset. The presets consist of: ‘happiness’,’sadness’,’anger’,’disgust’,’surprise’,’fear’,’neutral’,’clear’

Parameters

expression (str) – name of the desired facial expression (happiness, sadness,anger,disgust,surprise,fear,neutral,clear)

sendTrainedFaceExpression(expression)

Changes NICO’s facial expression to the given network predicted preset. These consist of: ‘anger’, ‘happiness’, ‘neutral’, ‘sadness’, ‘surprise’

Parameters

expression (str) – name of the desired facial expression (‘anger’, ‘happiness’, ‘neutral’, ‘sadness’, ‘surprise’)

send_PIL(Img, disp)

Send an image to a display. Can be “l” or “r” for left and right elbrow or “m” for mouth

Parameters

  • Img (PIL.Image) – Image to display

  • disp (str) – part of face to send (l, r, m)

send_bitmap_face(brow_left=None, brow_right=None, mouth=None)

Displays given bitmaps on the face

Parameters

  • brow_left (np.array (shape=(8,16))) – 8x8 bitmap to display as the left brow

  • brow_right (np.array (shape=(8,8))) – 8x8 bitmap to display as the right brow

  • mouth – 16x8 bitmap to display as the mouth

send_morphable_face_expression(expression)

Changes NICO’s facial expression to the given preset. The presets consist of: ‘happiness’,’sadness’,’anger’,’disgust’,’surprise’,’fear’,’neutral’

These presets are used for morph_face_expression. They are slightly altered versions of the regular face expressions.

Parameters

expression (str) – name of the desired facial expression (happiness, sadness,anger,disgust,surprise,fear,neutral)

send_polynomial_face(m1, m2, m_x_shift, m_crop_left, m_crop_right, left, l_x_shift, l_crop_left, l_crop_right, right, r_x_shift, r_crop_left, r_crop_right)

Generates and displays face with the given polynomials.

Parameters

  • m1 (list) – factor for each degree of the first polynomial mouth line

  • m2 (list) – factor for each degree of the second polynomial mouth line

  • m_x_shift (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • m_crop_left (int) – black border in pixels on the left of the resulting mouth image

  • m_crop_right (int) – black border in pixels on the right of the resulting mouth image

  • left (list) – factor for each degree of the left eyebrow polynomial

  • l_x_shift (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • l_crop_left (int) – black border in pixels on the left of the resulting eyebrow image

  • l_crop_right (int) – black border in pixels on the right of the resulting eyebrow image

  • right (list) – factor for each degree of the right eyebrow polynomial

  • r_x_shift (float) – reduces x values and therefore shifts curve to the right (x = -x_shift + i)

  • r_crop_left (int) – black border in pixels on the left of the resulting eyebrow image

  • r_crop_right (int) – black border in pixels on the right of the resulting eyebrow image

send_wavelet_face(m1, m2, left, right)

Generates and displays face expression with given wavelet paramters.

Parameters

  • m1 (tuple(float, float, float, float)) – wavelet parameters for first mouth line (stretch_in_y_position, offset_y_position, stretch_in_x_position , offset_x_position)

  • m2 (tuple(float, float, float, float)) – wavelet parameters for second mouth line

  • left (tuple(float, float, float, float)) – wavelet parameters for left eyebrow

  • right (tuple(float, float, float, float)) – wavelet paramters for right eyebrow

setCommMode(mode)

Sets the communication mode (0-2) the lower it is, the faster the higher it is, the higher the information from the arduino

Parameters

mode (int) – communication mode

show_PIL(img, scale=25)

Displays a PIL image

Parameters

Img (PIL.Image) – Image to display

sim_show_face()

Displays current face as image

trained_presets = {'anger': {'left': (-0.99951923, -0.00889372, 0.99983245, -0.14990053), 'mouth': ((0.99945402, -0.07992669, 0.99940026, 0.01424949), (-0.99829715, -0.11406033, 0.9997558, 0.04432757)), 'right': (-0.99873191, 0.08545645, 0.99995756, -0.04182587)}, 'happiness': {'left': (0.99983221, -0.07629592, 1.0, -0.04946393), 'mouth': ((-0.96794784, -0.01458586, -0.9989453, 0.00975196), (-0.95078206, -0.03179681, 1.0, 0.01479599)), 'right': (0.99992925, -0.03617397, 0.99996203, -0.01813084)}, 'neutral': {'left': (0.03521928, 0.0601279, 0.99998277, -0.05035896), 'mouth': ((-0.026799461, -0.50599956, 0.99360126, -0.01208178), (-0.025511968, -0.50718502, 0.99981982, -0.07333233)), 'right': (0.01149672, 0.0500899, 0.99979389, -0.07785152)}, 'sadness': {'left': (0.99999094, -0.03609413, 1.0, -0.05323452), 'mouth': ((0.99979699, -0.902700145, 1.0, -0.002130153), (0.99975657, -0.902467377, 1.0, -0.00777484)), 'right': (0.99998903, -0.06230563, 0.99999368, -0.01770263)}, 'surprise': {'left': (0.99999094, -0.03609413, 1.0, -0.05323452), 'mouth': ((0.99945402, -0.07992669, 0.99940026, 0.01424949), (-0.99829715, -0.11406033, 0.9997558, 0.04432757)), 'right': (0.99998903, -0.06230563, 0.99999368, -0.01770263)}}

CapacitiveSensors

The CapacitiveSensors class handles communication with the capacitive sensors inside the head of the NICO.

class nicoface.CapacitiveSensors.CapacitiveSensors(devicename=None)

Bases: object

CapacitiveSensors handles communication with the capacitive sensors inside the NICO head.

Based on contributions by Seed Robotics (www.seedorobotics.com)

_scan_ports(baudrate, timeout)

Automatically detects and establishes a connection with the FaceExpression Arduino

Parameters

  • baudrate (int) – Baud rate of the connection

  • timeout (float) – Read timeout value

getCapacitiveReadings()

Returns a list with the capacitive readings. Use the /len()/ python operator to determine the size of the list.

An empty object is returned if there is a communication failure or the feature is not supported on the Arduino sketch running in the head.

Returns

capacitive readings

Return type

list

recallibrateCapacitivePads()

Recallibrates the zero/baseline reading on the capacitive pads.

By default pads are callibrated on power up of the head board. Use this function only if you find significant deviations during operation.

validResponse(barray_readings)

Checks validity of received response

Parameters

barray_readings (bytearray) – response to verify

Returns

validity of the response

Return type

bool

Helper Classes

SerialConnectionManager

The SerialConnectionManager class allows shared access to the same serial device

class nicoface.SerialConnectionManager.SerialDevice(device, baudrate, timeout)

Bases: object

This class allows multiple objects to share the same serial connection.

_logger = <Logger nicoface.SerialConnectionManager (WARNING)>
_serial_connections = {}
close()

Detaches from the device, closes connection if no other object is attached

flushInput()

Flush input buffer of the connected serial device

static get_devices_by_manufacturer(manufacturer)

Returns all devices for a given manufacturer

Parameters

manufacturer – (partial) name of the manufacturer

Returns

list of devices

Return type

list(str)

reset()

Closes and reopens the connection to the serial device

send(message)

Sends a message to the device and returns the received response

Parameters

message (str) – message to send to the device

Returns

received response

Return type

str