Friday, March 23, 2012

Passing data to a custom component at runtime.

I have a question about what is allowed and what isn't its related to a raneg lookup question (http://forums.microsoft.com/msdn/ShowPost.aspx?PostID=95372) and engine threads (http://forums.microsoft.com/msdn/ShowPost.aspx?PostID=100596).

So one answer to the range lookup is to provide a script component that does the lookup. The challenge is obtaining the data for the lookup. You could write some code that took a sql connection or a flat file connection etc and read the data but that means you are fixed to the connection. Or you could do what I am about to suggest.

If I have a custom component that has 2 inputs. one to take the lookup values and one that takes the flow that contains the values to lookup.

Now the challenge is to get the lookup array populated before processing the other flow. My suggestion is to sleep the process input thread for the main data flow until the buffers for the other input has completed. So is the following good or bad ?

bool bLookupArrayLoaded = false;

public override void ProcessInput(int inputID, PipelineBuffer buffer)
{

bool bCancel = false;


if (inputID == lookupInputID )
{
//Do we have any rows
if (!buffer.EndOfRowset)
{
//Loop through the buffer to load the lookup array
while (buffer.NextRow())
{
//Populate the llokup array
}
}
else
{
//no more rows flag the array loaded
bLookupArrayLoaded = true;
}
}
else
{
//Has the array been loaded, if not sleep
while (!bLookupArrayLoaded ) Thread.Sleep (1000);

//process rows of the main data flow
if (!buffer.EndOfRowset)
{
//Loop through the buffer
while (buffer.NextRow())
{
//do stuff
}
}

}
}

You should not do this. If you mark your output as synchronous then your second input will never be called. If you mark is as asynchronous then it may or may not be called depending on the execution plan and more likely than not it will never get called, although you could test it out for every package you use the component it. I would strongly advise against this approach since it is a deadlock waiting to happen.

Thanks,
Matt|||Thats what I thought. But it works, is suspect because the input with the lookup values isn't synchronous with any output. So a seperate thread is used for that input and another for the other input.

I understand your point, it would be good if this was guaranteed as it means the component can use the power of the SSIS to load data with out the need to understand anything but an SSIS buffer.

I suppose this needs to be one of those items for Best Practice Analyser.

No comments:

Post a Comment