Pattern: External Callback

Context

Interfacing with external software systems involves not only calling externally implemented functions, but also implementing functions to be used by external systems (callbacks). We need a generic mechanism for implementing callbacks from other languages in Smalltalk.

Solution

Implement an ExternalCallback block which external systems can be passed like a function pointer to external systems to enable them to call back into Dolphin.

  1. Define any new external structures for any structure or pointer parameters you are going to use in the callback block which are not already defined.
  2. Define an argument type string describing the callbacks parameter types mapped to the appropriate external parameter types. The type string does not include the return type (the result answered by the block will be sent the #asLRESULT message for conversion to a 32-bit return value, the only possible return type at present).
  3. Define a block with the correct number of arguments for the callback procedure, inside which you implement your callback functionality. The block can contain whatever code is necessary to implement the callback. For many callbacks, the return value is important, e.g. to continue or terminate enumerations. The return value is the result of the last expression in the block, and should be an object with a conversion to 32-bit integer when sent #asLRESULT.
  4. You may need to add an external method to the appropriate external library function to enable you to register your callback function with the relevant external library. Use an lpvoid parameter type for the callback argument (the function pointer), and send the external callback the #asParameter message to persuade it to answer its machine code thunk. If the callback is for the use of a control (or some other type of window) which has a SendMessage() interface, then you may need to define an appropriate external structure for a parameter block, or you may have to pass the external callback directly as the lParam.
  5. Create an ExternalCallback instance using the #block:argumentTypes: message, passing (respectively) the callback block and the argument type description string. You must retain a reference to this external callback object to prevent it being garbage collected, as although the ExternalCallback class maintains a register of its instances, the register is a weakling.
  6. If a particular callback is frequently created, then consider using a precreated ExternalDescriptor (which holds a "compiled" representation of the argument type string) in conjunction with the ExternalCallback>>block:descriptor: instantiator. This technique is illustrated in the example.
  7. Pass your external callback object to the library method you defined, and wait for the callbacks to come pouring in!
  8. When you've finished with the callback, you can explicitly #free it. If you're not sure when you'll have finished with it, then simply leave it to be finalized when Dolphin garbage collects it (see Weak References and Finalization).

Example

An important callback example in the development system is that implemented for streaming text out of a rich edit control in RichTextEdit. (the browsers would not work without it). RichTextEdit actually defines two callbacks (one for streaming in, and one for streaming out), but that for streaming out is as follows: 

streamIn: aStream format: streamFormat
	"Private - Read text from the stream aStream.
	The receiver holds on the to the stream it is reading, because
	the RichEdit control appears to read from it asynchronously.
	Answer the number of characters read from aStream.
	Implementation Note: Extend the life of the old 'stream' to the end of the method 
	in case control still using it - BUT we must be very careful not to cause a reference 
	to the old callback to be kept in this method context, otherwise we'll build a huge 
	linked list of callbacks, blocks, and method contexts, etc."

	| answer callback text size |

	callback :=
		ExternalCallback
			block: [ :dwCookie :pbBuff :cb :pcb | 
					text := aStream nextAvailable: cb.
					size := text size.
					pbBuff replaceFrom: 1 to: size with: text startingAt: 1.
					pcb value: size.
					0	"The help is confusing/wrong. We must return 0 to continue streaming."]
			descriptor: ##(ExternalDescriptor argumentTypes: 'dword lpvoid sdword DWORD*').

	winStruct
		pfnCallback: callback asParameter yourAddress.

	answer := self sendMessage: EM_STREAMIN wParam: streamFormat lpParam: winStruct.
	self setModify: false.

	"It seems we have to increase the limit again after streaming in."
	self setMaxTextLimit.

	streamIn isNil ifFalse: [streamIn free].
	streamIn := callback.
	^answer

Notice how the arguments to the block are already objects of suitable types, and can be used directly, even the DWORD* parameter.

Like any other block, the callback block captures the environment in which it was created, so we can directly reference all the closure information we need (e.g. the argument to the method, aStream)

The interface to the rich edit control is SendMessage() based, and so uses a parameter block to hold the function pointer and cookie ("extra data" or closure information). We don't need the cookie, so we just pass 0, and ignore the argument in the callback block.

Known Uses