For those of you new to the product line, Intermediary is our on-premise API/Webservices/Security gateway product.
(In case you wonder why I do not cover Management Server and Agent, there is no supported upgrade path.
It is a fresh install, configuration components can be imported and you can migrate the database schema to still have audit data available.
Statistical data is then no longer accessible via the UI but still via the database directly.)
Back to Intermediary (AI), in version 8/9/10 versions we had three main concepts of configuration distribution:
- Local Export/Import on AI level
- Configuration replication via clustering
- Deployment Export Profiles
Let’s take a look at each option and how it works in the latest version:
Local Export/Import on AI level
This option can be used to backup the configuration or to move from one environment to another.
There is no change to previous versions.
We do not recommend to use this option for maintaining the configuration across AI instances in the same environment.
Configuration replication via clustering
This option no longer exists. There are only a few use cases left that require a cluster configuration.
Deployment Export Profiles
Export Profiles were already used by many of your customers in older versions. They now became the default/recommended approach for configuration distribution.
An Export Profile is a defined set of AI configuration components (e.g. Service Group, OAuth settings, Security Contract). It is persisted in AI and can be refreshed and uploaded to AMS. This is usually done each time there is a configuration change that you want to deploy to all your AI instances.
There is no in-place upgrade. You start with a fresh installation of Aurea Monitor and then launch an Intermediary profile.
The Launcher can either run as a Docker container or a simple process.
Once Intermediary started you might try to access the web console: http://localhost:4400/sst/
Tip: If it does not allow you to log in, verify that the user you used is set up and has the Admin role assigned ( http://localhost:4040/lgserver/admin/security/users/users_list.jsp).
If that is the case ensure that the user directory is set and configured on the used launcher profile ( http://localhost:4040/lgserver/admin/services/profile/profile_list.jsp).
Where to start?
Most customers have three configuration sources that we have to consider.
- Intermediary configuration components (e.g. Service Group)
- JVM arguments and system properties
- Tuning done via settings.jsp
In the latest version, you should configure everything on the profile level.
No configuration changes should be made on the Intermediary directly (except for the source AI, see below).
So where do all the settings go then? The below picture outlines the migration and configuration flow. The next sections will explain this further:
First, go to your old Intermediary (8/9/10.x) and export all your configuration components to the local disk (e.g deploy.zip).
Next, you import everything on the new Intermediary (11.x+).
Tip: Use dedicated transports for your access points. Ideally do not reuse the default http/https listeners.
This can cause issues with different profile configurations (e.g source and target profiles have different default listener ports).
Now you should take some time to think about the packaging/grouping of your services/APIs.
You can group them by business-service and/or by the load they have to handle. The grouping is done via dedicated Export Profiles.
The way you define this grouping defines how easy it is to scale later.
You have to ensure that each Service Group is only added to one Export Profile.
It is not supported to assign multiple Export Profiles with the same Service Groups to an Intermediary Launcher Profile.
Tip: On the selection of the exported components ensure that you have “Always the active revision”
Once all the Export Profiles are defined you have to upload each to the Management Server.
All further configuration is done on the Management Server, where you can find all the uploaded Export Profiles (http://localhost:4040/lgserver/admin/deployment/distpackages/exportedartifacts_list.jsp).
Launcher Profile creation and/or configuration is the next thing you need to do (http://localhost:4040/lgserver/admin/services/profile/profile_list.jsp).
You might want to keep it simple and simply reuse the default_intermediary_profile for all your AI instances.
That is ok, but might not be ideal.
Let me explain why I think that you should at least create one additional Launcher Profile:
Each time you make a configuration change or create a new Service Group the Export Profile has to be uploaded to the Management Server. Then provisioned to the AI instances.
If you have only one Launcher Profile, the AI instance that uploaded the configuration will also get the changes provisioned, potentially replacing changes that were not uploaded yet.
To avoid this I recommend having a (or up to one per Export Profile) dedicated “source” AI instance with a dedicated Launcher Profile.
You can do local imports (e.g. from a lower environment), create a new configuration on it, upload to the server, and then provision to other (“target”) AI instances.
As you (hopefully) have already split your configuration into multiple Export Profiles you can spread these now across different (“target”) Launcher Profiles.
Or just one if your configuration is not complex, you want to keep it simple and have no service-specific scaling requirements.
Let’s assume that I have convinced you that a dedicated Launcher Profile makes sense.
So we create now a new Launcher Profile and configure it:
Launcher Profile Configuration
There are several sections but I would like to call out a few:
- Additional JVM Arguments
- Additional Packages
- User Directory
- Listener Settings & HTTP/S Listener
- Management (Database)
- Provisioning Packages
Additional JVM Arguments
This is an important section.
Any setting/system-property that was previously defined in your AI local env.sh/post-custom.conf/ActionalIntermediary.conf or your settings.jsp goes in here.
Please compare them to the settings you defined in the AMS post-custom.conf. If these do not match then you will not be able to provision the AI instance.
This section defines which packages the launcher should download.
Once downloaded they are either started (e.g. Intermediary) or used by the main package (e.g. SonicMQ JMS libraries).
You can define your own packages to provide additional libraries (e.g. Saxon for XSL 3 support on Intermediary) or custom Java Agents.
In previous versions, the users were managed locally on Intermediary or you configured on each instance a third party (e.g. LDAP).
Now, this is also managed centrally. You can either use the Local Users from AMS (http://localhost:4040/lgserver/admin/security/users/users_list.jsp ) or you can configure centrally an External User Directory (http://localhost:4040/lgserver/admin/security/directory_services/externaldir_list.jsp ).
Listener Settings & HTTP/S Listener
The listeners you configure here replace what you did in version 8/9/10 via “http://localhost:4400/appsrv/”.
This is the Jetty listener configuration and not the AI HTTP/S transport configuration.
Transport configuration is still done on the (“source”) Intermediary console and then uploaded to the AMS using the Export Profile.
You have to ensure that the ports used in the AI transports match a Jetty listener of the Launcher Profiles.
Dedicated transports (see the first Tip) for access-points are highly recommended.
Let us go through an example:
|Source Intermediary (source_intermediary_profile)|
|Transport (AI console)||http/4400|
|Service Group||SG1 using http on AP1|
"http" is the default HTTP listener and automatically mapped to the first Jetty HTTP listener you configured.
Usually, this is the port you use to access the web console of AI. If you now add SG1 to an Export Profile and upload it, it will reference the default transport with the name "http".
Then you assign this Export Profile to a different AI profile where Jetty listeners are defined for 5500 and 5501. If you start this instance and try to access port 5501 you will get:
Admin and WebServices Viewer Console access is not allowed on runtime-only transport
The reason is that there is a Jetty listener but no HTTP transport to handle the request. The only transport that exists by default is “http” which is in this case mapped to 5500.
Now you might think that 5500 will still work but it does not work either.
Jetty listens on the port but the http transport got overwritten by the Export Profile with port 4400.
The result is this:
|Target Intermediary (target_intermediary_profile)|
|Jetty Listener||5500, 5501|
|Transport (AI console)||http/4400|
|Service Group||SG1 using http on AP1|
To avoid this problem you should use dedicated (non-default) HTTP transports for your access-point configuration.
The other option is to not add the default transport to the Export Profile but then you cannot do any configuration changes on the HTTP transport (e.g. Service Location URL).
Configure the database to be used by Intermediary. Usually, this is the same as the server database.
Now we are back to the Export Profiles which were created earlier.
You can assign one or more Export Profiles to the Launcher Profile.
Each AI instance is assigned to a given Launcher Profile.
Most customers had one large Intermediary instance per server. This does not really scale well.
If one service was used a lot then usually another node/server was introduced and added to the Intermediary (cluster) setup.
Now with version 11.x+ it is way more attractive to have more but smaller instances.
You can create one Launcher Profile per Export Profile and this way you can scale on the Export Profile level.
You can manually configure how many instances of each Launcher Profile you run.
Or you can make one more step and use containerization (Docker) together with a container-orchestration system like Kubernetes.
This will give you stateless containers (everything is configured on the profile level) and dynamic scale.
Putting enough thought into defining the Export Profile level granularity is key to enable the full potential of this.
This post became a little bit longer than I thought it would. So let’s just recap what needs to be done:
- Make a fresh install of AMS
- Configure the launcher profile
- Export the complete AI 10.x configuration
- Import it on an 11.x AI
- Define Export profiles and upload them to AMS
- Assign them to the target profile(s)
- Launch Intermediary instances
Feel free to leave a comment or log a support case if I totally confused you by now or there are any questions left.