How to implement IAsyncResult using compiler extensibility
In previous posts I gave an introduction to Boo and the extensibility of the compiler. In this post we are now able to look at how compiler extensibility can be used to improve the creation of asynchronous operations.
Initially, I thought it would be straighs't forward to modify the model created earlier with text templates for use in compiler extensions. I did run into some interesting problems.
First, I realized that the model I had created was really two integrated models. One for capturing metadata about API exception behavior, and one for modeling the IAsyncResult design pattern. Since my goal was to model the IAsyncResult design pattern, I dropped the portions of the model related to exception handling.
Second, I quickly found that the syntax I wanted to use was not legal Boo syntax and had to change strategy. My first syntax attempt was something like this:
operation Send(host as string, port as int, buffer as (byte), offset as int, size as int): MyClass
MyClass is the class on which the asynchronous operation would be implemented. The problem is that this is a C# notation, and the parser was not happy.
The solution
The solution that seemed more compatible with Boo turned out to be as follows:
partial class MyClass:
operation Send(host as string, port as int, buffer as (byte), offset as int, size as int):
m_socket as Socket
def Helper(something as bool):
print something
step Connect:
m_socket = Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
m_socket.BeginConnect(m_host, m_port)
step Send:
m_socket.BeginSend(m_buffer, m_offset, m_size, SocketFlags.None)
step Disconnect:
m_socket.BeginDisconnect(true)
By nesting the operation in the class it was to be applied, the macros are now able to find the class where to add the BeginXXX and EndXXX methods by navigating to its parent in the AST.
In addition any additional member variables or methods (e.g. m_socket and Helper) can be defined within the operation and copied to the class derived from AsyncResultNoResult.
You can see some other aspects of the syntax if you look closely:
- The name of the operation appears as the method name after the operation declaration. Send in this example.
- Arguments passed into the Begin method are specified as arguments to the operation
- The userCallback and state parameters are not specified in the BeginXXX calls within the steps. These are auto generated method names and are inserted automatically for you.
- The step macro specifies the name of the Step, which is used to construct generated callback methods.
Step macro
Because macros are evaluated innermost to outermost, the step macro will be evaluated before the operation macro. Since the step macro can’t render all that it needs to when evaluated, it saves its information in the operation macro for when it is evaluated. Its implementation is as follows:
macro step:
nameExpression = step.Arguments[0] as ReferenceExpression
stepName = nameExpression.Name
d = operation["steps"] as List[of Step]
if d == null:
d = List[of Step]()
operation["steps"] = d
stepBody = step.Body
d.Add(Step(stepName, stepBody))
return null
As you can see it is very simple. It saves the name and body of the step in a collection of Steps on the operation macro for evaluation later. It then returns null to remove itself from the AST.
Operation macro
The operation macro first creates the BeginXXX and EndXXX methods on the class it is extending. Since the operation macro is nested within the class it is extending, navigation of the AST will get the class declaration it needs to extend:
macro operation:
# Get the class definition and class name
if operation.ParentNode != null:
definition = operation.ParentNode.ParentNode as ClassDefinition
else:
definition = null
if definition == null:
# There must be a better way to fail the compilation than this.
assert false, "operation must be used within class definition."
return null
className = definition.Name
# Keep the module to add classes to
module = definition.ParentNode as Module
# Extract the name of the method and arguments.
methodExpression = operation.Arguments[0] as MethodInvocationExpression
assert methodExpression != null
nameExpression = methodExpression.Target as ReferenceExpression
assert nameExpression != null
shortName = nameExpression.Name
resultClassName = nameExpression.Name + "AsyncResult"
# This prevents the method name from being 'quoted'. Probably not the right node type
# but I believe it may get turned into a string and reparsed anyway.
methodReference = ReferenceExpression(resultClassName)
# Build a partial class to hold Begin/End methods
extendedClass = [|
partial internal class $className:
def $("Begin" + shortName)(asyncCallback as AsyncCallback, state as object):
result = $methodReference(asyncCallback, state, self, $(shortName))
result.Process()
return result
def $("End" + shortName)(result as IAsyncResult):
AsyncResultNoResult.End(result, self, $(shortName))
|]
At this point, we have the class name and a prototype class with BeginXXX and EndXXX members. From the AST assigned to extendedClass, we can extract the members and add the arguments to the BeginXXX method. Note that the arguments are also passed to a prototype constructor of the AsyncResult derived class so it is done in the same loop. Here is the prototype for the asyncresult class:
# Build a template for the derived AsyncResult class
resultClass = [|
partial internal class $resultClassName(AsyncResultNoResult):
internal def constructor(
asyncCallback as AsyncCallback,
state as object,
owner as object,
operationId as string):
super(asyncCallback,state,owner,operationId)
internal override def Process():
pass
|]
Here’s the code that finds the constructor and the begin method, and appends the arguments to the prototypes of the methods:
# Find Begin method which needs additional arguments
beginMethod = extendedClass.Members[0] as Method
assert beginMethod != null
# Find the constructor call on right side of assignment statement
es = beginMethod.Body.Statements[0] as ExpressionStatement
assert es != null
be = es.Expression as BinaryExpression
assert be != null
mie = be.Right as MethodInvocationExpression
assert mie != null
# Find the constructor which needs additional arguments
ctor = resultClass.Members[0] as Constructor
assert ctor != null
for a in methodExpression.Arguments:
# Add parameter to begin method
beginMethod.Parameters.Insert(-2, ArgumentToParameterDeclaration(a))
# Pass the parameter through to the constructor
mie.Arguments.Insert(-4, ArgumentToPassedArgument(a))
# Add parameter to ctor
ctor.Parameters.Insert(-4, ArgumentToParameterDeclaration(a))
# Add field to result class
resultClass.Members.Insert(-2, ArgumentToField(a))
# Add field assignment to constructor
ctor.Body.Statements.Add(ArgumentToAssignFieldStatement(a))
In addition to adding the arguments, it also adds the fields and assignment statements for the parameters of the constructor to the fields of the result class in the constructor body.
Building the steps of the AsyncResult
Building the sequence of steps necessary to do the asynchronous operation is similar to how it was done using text templates in my earlier post. The first step is started in method called Process on the AsyncResultNoResult derived class, and no exception handling is added.
Each subsequent step starts the following step when its completed handler is called. Each completed handler needs exception handling to make sure that Complete() is called on the result class on failure. The very last step also has to call Complete() on success.
The following code achieves this:
first = true
processMethod = resultClass.Members["Process"] as Method
previous as Step = null
method as Method = null
steps = operation["steps"] as List[of Step]
for step in steps:
if first:
first = false
for s in step.Body.Statements:
processMethod.Body.Statements.Add(s.CloneNode())
else:
ts = method.Body.Statements[1] as TryStatement
assert ts != null
for s in step.Body.Statements:
ts.ProtectedBlock.Statements.Add(s)
resultClass.Members.Add(method)
previous = step
method = step.GetCompletedMethod(false)
method = previous.GetCompletedMethod(true)
resultClass.Members.Add(method)
Note that steps beyond the first have a try/ensure/finally inside the handler where the statements are added. The first step does not have exception handling so the body statements are added directly to the Process() method.
Copying member and field declarations
In our DSL we allow developers to declare fields and methods that should be included on the result class. In this case it was m_socket and a method called Helper(). After the step macros were expanded, they returned null to remove themselves from the operation body. That means everything that remains should be copied to the result class as is. The following code does just that:
# Add member statements to resultClass
for s in operation.Body.Statements:
decl = s as DeclarationStatement
assert decl != null, "Only declarations are supported in operation body."
c = [|
class temp:
$(decl)
|]
for m in c.Members:
resultClass.Members.Add(m)
Since both fields and methods both are DeclarationStatements, the code copies only DeclarationStatements to the result class for simplicity.
Finishing up
Once the result class has all its fields, methods and the arguments are added to the constructor, the class needs to be added to the model. The Begin and End methods need to be added to the hosting class, and we need to remove the operation macro from the AST by returning null. This is done by the few lines shown below:
# Copy the Begin/End methods to the class the operation state was nested in.
for m in extendedClass.Members:
definition.Members.Add(m.CloneNode())
# Add the result class to the module
module.Members.Add(resultClass)
# Everything has been added to the module or class
# so remove the node.
return null
Summary
In this post, I finally got to show you how you can use some macros in Boo to extend the compiler to support asynchronous results. The solution takes a little while to understand but once you have extended the compiler, the compiler supports a pretty simplified syntax for implementing asynchronous operations. Making things simple should improve productivity.
Previously we evaluated using text templates with the code DOM for generating code. In the next post, I will review how text templates compare with compiler extensibility as a mechanism for generating repetitive code.
Series
Start of series previous next