OrsMolmoVisualAssistant_6ab457dae33011ef8c4708bfb84fea76

author:

contact:

email:

organization:

address:

copyright:

date:

Feb 04 2025 14:44

dragonflyVersion:

2024.2.0.1810

UUID:

6ab457dae33011ef8c4708bfb84fea76

Class Code

class OrsPythonPlugins.OrsMolmoVisualAssistant_6ab457dae33011ef8c4708bfb84fea76.OrsMolmoVisualAssistant_6ab457dae33011ef8c4708bfb84fea76.OrsMolmoVisualAssistant_6ab457dae33011ef8c4708bfb84fea76(*args, **kwargs)
classmethod getMainFormClass()

Gets the class of the main form

Returns:

output –

interfacemethod multiROIVisualAssistant(viewGenealogicalName: str, generatedText: str, inputChannel: Channel, sliceChannel: Channel, sourceBoundedPlane: Rectangle, timeStep: int) → MultiROI

Create a multi-ROI using the outputs of the Molmo model

Parameters:

  • viewGenealogicalName (str) – the current view’s genealogical name

  • generatedText (str) – the generated text from the model’s output

  • inputChannel (ORSModel.ors.Channel) – the current channel

  • sliceChannel (ORSModel.ors.Channel) – the channel adjusted for the view

  • sourceBoundedPlane (ORSModel.ors.Rectangle) – the plane adjusted for the view

  • timeStep (int) – time index

Returns:

output (ORSModel.ors.MultiROI) – Returns all the output points from Molmo as a multi-ROI

interfacemethod pointsVisualAssistant(viewGenealogicalName: str, generatedText: str, sliceChannel: Channel, sourceBoundedPlane: Rectangle, timeStep: int) → Annotation

Create point annotation using the outputs of the model

Parameters:

  • viewGenealogicalName (str) – the current view’s genealogical name

  • generatedText (str) – the generated text from the model’s output

  • sliceChannel (ORSModel.ors.Channel) – the channel adjusted for the view

  • sourceBoundedPlane (ORSModel.ors.Rectangle) – the plane adjusted for the view

  • timeStep (int) – time index

Returns:

output (ORSModel.ors.Annotation) – Returns all the output points from Molmo as point annotations

interfacemethod runVisualAssistantModel(viewGenealogicalName: str, inputChannel: Channel, userInput: str, progress: bool = True) → Tuple[str, Channel, Rectangle]

Run the Molmo model and generate a response

Parameters:

  • viewGenealogicalName (str) – the current view’s genealogical name

  • inputChannel (ORSModel.ors.Channel) – the current channel

  • userInput (str) – the user input to Molmo

  • progress (bool) – True or False for creating a progress bar

Returns:
classmethod startupDefault(openMainForm=True)

Creates an instance of the plugin class and opens the main form (if required) at the default position.

Returns:

output (AbstractPlugin) – plugin instance

interfacemethod unloadVisualAssistant() → None

Unload Visual Assistant from VRAM/RAM