Part 4 – Creating the Incident via the Remedy Email Engine

Back here I started this whole overview on automating Remedy using the Remedy Email Engine.  Part 2 and Part 3 followed.

Here in Part 4 we are going to get to the meat of it.  Up to this point we have done all the work necessary to get emails into our system and into our custom email processing form.

So now what?

Creating an Incident from an Email

Now we are at the best part – creating a new Incident.  Since we’ve already processed/formatted everything this part is very straight forward.  Examine this image:

Push to Incident (click for full image)

Push to Incident (click for full image)

One of my most major complaints about Remedy is that you can not open the field mappings box to see more than a few lines….that said…the mappings in this case are very simple.

In previous versions of BMC Remedy Help Desk you would simply push the value to the form.  With the advent of BMC Remedy Incident Management you can no longer do it.  Well, you CAN do it – it’s just not very fun.  Instead, we are going to use the built-in interface for creating Incidents.  The advantage here is the number of fields we need to map is quite minimal.

In short, we are pushing from our current processing form to the HPD:IncidentInterface_Create form.  The mappings are as follow:

  • Status – New (List value)
  • Description – $Subject$ (email subject line)
  • Assigned Support Organization = “My Organization” (a valid support organization – you can hold this in your configuration form and do multiples if you want to get fancy)
  • Last Name – “Caller” (Static value in my case due to unpredictable callers.  This could easily be looked up and configured to a real/existing person’s information)
  • First Name – “Unknown” (Static value in my case due to unpredictable callers.  This could easily be looked up and configured to a real/existing person’s information)
  • z1D_Action – CREATE (Mandatory field value to create an Incident)
  • Service Type – User Service Restoration (List value)
  • Detailed Description – This one gets complicated.  There are carriage returns inserted below everywhere you see a & sign.  I had to replace them to make everything happy on the web:  (((((((((((( “To: ” + $To:$) +  ” & “) +  “From: “) + $From:$) +  ” &”) +  “Date: “) + $Create Date$) +  ” &”) +  “Subject: “) + $Subject:$) +  ” &”) +  “Message:”) + $Message Body$
  • Urgency – 4-Low (List value)
  • Impact – 4-Minor/Localized (List value)
  • Reported Source – Email (List value)
  • Assigned Group – “Support” (Configuration value or static depending on your needs)
  • Assigned Support Company – “My Company” (The appropriate company to go with the support group)

When this push fields happens an Incident is created – and that’s it.  In our case the agent then goes on to examine the email and respond to the customer appropriately.  The agent may also update the Caller/Contact information with a real caller.  They can do this either by selecting an existing one or creating a new profile for the caller.

If this is done the caller will even get updates to the Incident via email (if configured correctly).  This brings up to our next concern – how do we update Incidents via email?  That is in part 5.


Part 3 – Backend Workflow used to Automate Incident Creation via the Remedy Email Engine

This is part three of a series of posts about automating service with the Remedy Email Engine. 

Part one covers the overall general concept and big picture considerations.

Part two covers technical considerations dealing with AR Server type, email protocols, and similar concerns.

This post will cover the first (and easiest) portion of the coding – namely, getting the email into a temporary processing form where we can process it.

There is a basic assumption for this post: You have already created an incoming email address for the Remedy Email Engine to access.  You have also determined the protocol it uses (MAPI, POP3, etc) and tested the connection to make sure the Remedy Email Engine connects and retrieves the messages and puts them into the ”AR System Email Messages” form.  This is all pretty straight forward stuff.

Creating the Processing Form

Before anything else is done a temporary form must be created for processing the emails.  We could process emails using filters attached to the “AR System Email Messages” form but that would be customizing base product code which I try to avoid at all costs.  It’s simply not worth it.  Instead, I create a custom form to do the processing and attach one filter to the “AR System Email Messages” form to push the data over.  We’ll get to that filter later – first we need somewhere to PUT the data.

This is a pretty simple exercise.  First, we’ll just create a standard form.  I prefer to copy the BMC Remedy ITSM look and feel for my forms just to keep everything consistent.  This particular form might be accessed manually by an administrator from time to time so I added a nice consistent look and feel.  Then I added a page holder with three pages:

  • Custom Email Fields
  • Auto Populated Fields
  • Core Fields

Custom Email Fields

On the Custom Email fields tab I added the fields in this screenshot:

Custom Email Fields tab

Custom Email Fields tab

 The fields are self explanatory but I’ll cover it for the sake of clarity:
  • To – The original “To” value from the email
  • From: – The original “From” email address value
  • Subject: – The original email subject
  • Message Body: – The original email address
  • Delete On: – Date to delete this message on automatically.  More on that later….
  • System Matching Address: The address used in the system to validate the “To” field.
  • Status: The status of this email.  Possible values include the following
    • New
    • Processed
    • Invalid

I’d like to make just a couple of quick notes on these. 

The “System Matching Address” is set via basic workflow and checks a configuration form.  All it does is a “Set Fields” action that says “Tell me what ‘To’ address is valid”.  This prevents internal spam.  If your company is XYZ company and the valid address is Support@XYXCompany you can add workflow to negate other addresses and mailing lists like AllEmployees@XYZCompany.  In those cases the email ‘Status’ is set to “Invalid” at this point so this email does not get processed and create an Incident.  You can of course make this work a number of different ways.

The ‘Status’ field is set to “New” when a record is created (Submit).  It is set to the value “Processed” when an email successfully creates an Incident.  It is set to “Invalid” when you code for internal spam as I explained above.

Auto Populated Email Fields

We also have a tab called “Auto Populated Email Fields”.  This contains the fields shown in this screenshot:

Auto Populated Fields

Auto Populated Fields

 These fields are necessary for the creation of the Incident using the HPD:IncidentInterface_Create form.  All of these values are necessary.  In my instance we had quite a few people contacting us who were unknown – we did not have Customer (People) records created for them in the Person form.  Nor could we predict the Support Company, Organization, or the appropriate assigned group.

Fortunately there is a very easy solution to this – and you could do something similar.  I created a configuration data form that held the default values for these.  They were different per server since our development server used different values than our production server.  

A sample of the values that I put into this form were:

  • First Name: Unknown
  • Last Name: Caller
  • Support Company: My Company
  • Support Organization: My Organization
  • Assigned Group: Support

“Unknown Caller” was an actual Person record in the system.

These are static values and of course you could easily add workflow at this point to look up the Person record based on the “From” email address.  That would be a cleaner solution for most organizations if their callers are known.  If you are often creating new customer profile (Person) records the solution I used may work for you – after all, once the Incident is created the agent merely has to read the detailed description for the Incident and Create/Select the appropriate person record.  This is far less work that manually creating the Incident.

Core Fields

The Core Fields tab is self explanatory – the Regular Form core fields are parked here.  I rarely use the core fields for a variety of reasons but you’ll notice in this case I did use ‘Status’ on another tab.  The rest are defaulted to standard values.  I’ll leave that at that:

Remedy Core Fields

Remedy Core Fields

Workflow on the AR System Email Messages Form

There are actually two things we do with incoming email messages – for now we’ll simply cover a “Create” situation where an email comes in and we want to create a new Incident.

In this case the “Run if” is blank – we push all incoming messages to our processing form.  It’s easier to process them there.

There’s also a more important reason to not have a “Run If” when pushing from the “AR System Email Messages” form.  The Java process that runs the email engine will fail to create an entry in the “AR System Email Messages” form if the “Run If” fails.  So, it’s better to insure the messages come in correctly and then deal with them in another place.

So, the workflow to push to our processing form is pretty straightforward:

Push from AR System Email Messages

Push from AR System Email Messages

As you can see this simply pushes messages into the processing form.  From there the workflow I mentioned before sets the “Auto Populated Fields” values.

In the next installment I’ll cover the nuts and bolts of creating an Incident.  After that I’ll go over the trickier subject of updating 


Part 2 – Technical/Architecture considerations for the Remedy Email Engine

Technical Considerations of automating service with the BMC Remedy Email Engine

This is a continuation of a previous post which can be found here.

There are a few technical considerations that must be addressed before you get down into the dirt creating this type of integration.

Email Server Protocol

Many Remedy customers - especially at smaller companies – don’t have to worry about this.  By far the most popular implementation of BMC Remedy products with small to mid-market companies is the following configuration (versions notwithstanding):

  • BMC Remedy AR Server on Windows
  • Microsoft SQL Server on Windows
  • Assorted tools, etc, also running on Windows

Often these are all on the same server – and there is simply nothing wrong with that.  If you have a call center processing a few hundred calls a day this is more than enough to sustain your service management practice.  And as it stands this makes automating the Remedy Email Engine very simple – you can run the BMC Remedy Email Engine  and use the MAPI protocol to communicate with Exchange.  You can literally configure this and have it up and running in mere minutes.  I’ve done this many times myself.

However – this does not apply to everyone.  I typically work with large customers like the US Federal Goverment (civilian agencies) and Fortune 500 companies.

Many of these customers are taking thousands of calls per day and consequently are using more robust hardware.  Typically this is something similar to the following configuration:

  • BMC Remedy  AR Server on Solaris
  • Oracle 9/10 on Solaris (remote server)
  • Assorted tools (running on the AR Server)
  • Other remote mid-tier servers, etc
  • MS-Exchange running on Windows servers completely separate from the AR Servers

Here’s the problem.  MAPI is strictly a Windows protocol used to send and recieve email.  You can not use the MAPI protocol to check email from a Solaris server running the BMC Remedy Email Engine.

Of course – there are other protocols – specifically POP3-  which works on nearly every platform and operating system.  It was after all the original protocal used in most internet-based email applications and is still popular today.  POP3 is only used to receive email from an email account so that only gives us half of the equation.

And of course – there’s even a problem with that.  It is not considered secure.  Since I often work at sites that needs security in all aspects of the system we had to get special wavers AND an entire separate exchange server for our email accounts that were accessed through POP3.  Along with that came security requirements that restricted access to those email servers for any IP except our BMC Remedy Email Engine server.  All of this tooks numerous forms, meetings, phone calls, etc.

So the lesson here is simple: Don’t assume that setting up the Remedy Email Engine is going to be simple just because it is technically a breeze.  Security and policy considerations can be quite restrictive.

On the “sending email” from the BMC Remedy Email Engine we did luck out on the project though – the customer maintained a corporate “outbound” email server that was used for all automatic sending of emails and did so with few security restrictions.  You will have to investigate the rules for your organization but if you are on Solaris and using Exchange (or a similar combination) the only real option for sending email is SMTP.  The ways to do this are too numerous to bother listing but they all qualify as “easy”.

Accepting/Sending emails outside your domain

There are two policy issues you will also want to consider before you begin coding.

First, will you accept email from outside your domain?  It’s fairly easy to immediately delete every incoming email that’s not from MYCOMPANY.COM – but is it safe?  If your CEO is working from home and uses his Gmail account (I have literally seen this one happen) to request support do you want to ignore it?  Maybe – it depends on your corporate policy.  You also may have to deal with fighting spam if you are accepting emails from the entire world.

Second, will you send emails outside your domain?  If you are requesting any type of sensitive information (however YOU define it…) you may want to limit how far out in the world you are willing to send emails.  Or just for the sake of efficiency, company policy, etc.

It is better to decide both of these items up front.

Fighting Internal Spam

Another consideration is how to deal with internal spam.  Nearly every company I have worked with has company-wide (or at least site-) email distribution lists that are used for things like “It’s Karl’s birthday! Come and get cake!” or other public announcements.  Naturally you don’t want these to generate Incidents.

The best way to do this is to prevent the email account from being included in that sort of thing from the beginning.  That is impossible for many organizations however and that means you should have a plan “B” to deal with internal spam.   Here are some suggestions in no particular order:

  • Allow agents to immediately close Incidents that are obvious spam without working on them
    • Pros – No additonal coding
    • Cons – It’s sort of a tradition in call centers for management to not trust agents to immediately close things ever.
  • Code for the known/likely internal spam types
    • Pros – It’s easy to predict the daily “Company Update” email
    • Cons – There could be N^100000 of these; also, they could change at any time
  • Tag messages withtext and look for it in workflow
    • Pros - Very easy to do with form-launched or other pre-formatted emails, templates, replies, etc
    • Cons – The “better mousetrap” has never beaten the “more stupid user” scenario

There are many multiple ways of dealing with this and it’s likely every organization will need to do their own planning.

Next, we’ll get on with the actual technical bits. Go on – read part 3 of this series!

-William Rentfrow, Principal Consultant, StrataCom Inc


Part 1 – Automating service with the BMC Remedy Email Engine

I (along with others) just finished an initiative at a customer site.  This mini-project was part of a larger implementation.  The entire goal was to create an interface that allowed the following to happen:

  1. Allow customer inbound emails to create an Incident in Remedy Incident Management 7.x
  2. Respond to customer case-creation emails with a standard template that listed further information the customer should provide, and
  3. Allow replies to the notification email to update the Incident

None of this is very difficult of course but before I get into a technical discussion about this it is worth noting WHY this is worth doing.

It’s plainly obvious that automation of any manual process is a good idea.  It removes mistakes, ensures consistency, and generally – and most importantly – saves money. 

In  this case all of these things are true.  And we can actually quite easily show the ROI from performing these relatively simple improvements.

For this customer I know that two people were working full time every day processing emails.  They were both full time, so that is 80 hours/week or approximately 4000 hours/year.

I have no idea what the staff in question are paid but I am aware of industry averages for similar positions.  For the same of simplicity let’s estimate they cost the company about $30/hr ($22/hr + payroll, insurance, etc…).  So the total cost of the work they performed is approximately $120,000 per year total.

If you total the coding, testing, training, etc time we spent on this project we are far, far under that amount.  In fact I don’t believe we even hit 100 hours.  Our internal pricing is pretty competative but even on the open market this project makes sense – you have immediate ROI and consistent delivery.

In short – just this one small project is going to save over $100,000 this year and improve service delivery.

There are some straightforward design considerations to talk through as well – you can find that in part two.

-William Rentfrow, Principal Consultant, StrataCom Inc


Remedy Development Best Practices

In response to:

** Hi folks.  Just wondering what practices some developers are using for custom code and enhancing ootb workflow.  IE, field ID’s in a non reserved range, New workflow easily identified perhpaps by the company name etc.  My real big dillemma is modifying existing workflow. Does anyone have some best practices for modifying existing workflow so that in the event of an upgrade pitfalls can be avoided or at least anticipated?

I am kind of a nut about this since 7.x came out.
 
Fields: Custom numbering in a predictable range.  Generally when customizing an existing form I start with something like 600100200 for the field ID’s.  Using this method you can easily run a simple SQL search on the form and field table to determine what your custom fields were:
 
For field names I use BMC’s naming convention which is pretty well documented.  I do not remember if they only give this out to partners or not.  If you can’t find it let me know.
 
For workflow and forms I use a special multi-tenancy way of naming.  It’s pretty straight forward.  If I am customizing original workflow I disable the original and never modify it.  Then I rename it with the Business Name:Operating Company Name or abbreviation: then original name.  So an Active Link named HPD:DoSomeStuff for XYZ Corp’s HR customer/tenant might get named XYZ:HR:HPD:DoSomeStuff.  Cumbersome?  Maybe. Useful? VERY – especially when you can narrow your view by prefix to find workflow specific to certain tenants.
 
I do this now even if the system is not planned to be multi-tenancy because you never know….
 
When making new workflow it’s similar – I take the form prefix (if one exists) or invent one if necessary for new forms.  The result is the same as the above.
 
When I create brand new forms from scratch I do not worry about numbering the fields.  An entirely custom form should never need have it’s fields differentiated from base product fields.
 
The only area I really run into a hitch philosophically is modifying base product active link guides or filter guides.  You can either
 
1.) Choose to disable, copy, rename, and modify the calling Active Link/filter, copy the entire guide, disable all the base product workflow, copy it, rename it, re-enable it, and customize the active link/filter in question, or
2.) Disable the active link/filter in question, leave it in the base product guide, add the new customized active link/filter, and document the heck out of it.
 
I pragmatically chose #2 and stuck with it based on what mood I was in that day.
 

William Rentfrow

Principal Consultant, StrataCom Inc.

www.stratacominc.com

wrentfrow@stratacominc.com

715-410-8056 C

715-592-5185 O