Every SAP IDM migration project eventually reaches a point where all workflows in the new IAM have been reviewed, all interfaces have been tested, and all the useful side jobs which have never been core to IAM, but which SAP IDM used to handle so effortlessly, have been moved to someone else’s scope. At that point, you might ask yourself: “How do we get our actual data from SAP IDM into the new system?”
In this article, I will discuss IDM CSV Server, a data migration tool for use with SAP Identity Management which I have worked on for the last couple of months. IDM CSV Server is fully agnostic of which specific target IAM vendor or product you have opted to migrate to. To emphasize that this discussion is applicable to any migration target, I’ll use „NextGen“ from here on as a generic alias for referring to your next generation IAM solution.
DISCLAIMER: IDM CSV Server is free and open source available under an Apache License 2.0. It is not endorsed or supported by SAP. I am not an employee of SAP or affiliated with SAP in any other way.
IDM CSV Server’s releases, documentation, source code and bug tracker can be found on the project’s landing page at:
https://codeberg.org/freicoder/idm-server-csv
IDM CSV Server’s latest release is always available for download from the the following stable link:
https://codeberg.org/freicoder/idm-server-csv/releases/latest
IDM CSV Server Architecture
The following block diagram illustrates IDM CSV Server’s architecture in the context of SAP IDM and NextGen:
The key point in IDM CSV Server’s architecture is that it provides a direct and secure communication path between SAP IDM and NextGen. Using any RESTful client such as the one included in IDM CSV Server’s distribution, NextGen pulls new data from SAP IDM via HTTPS whenever it needs to. Requests are initiated by the client, running on or near NextGen, and served by SAP IDM’s REST API, running on SAP NetWeaver.
In contrast to IDM CSV Server, traditional data migration schemes often require:
- direct “database sync” connections initiated from NextGen to read from the SAP IDM Database
- export CSV to SAP IDM runtime file system; forward to NextGen via file server or SFTP
- ad-hoc queries from developer workstation; forward to NextGen out-of-band/manually
Unlike many of these schemes, IDM CSV Server neither requires that your SAP IDM Runtime take on a file server role, nor does it require opening firewalls for direct access from NextGen to SAP IDM Database and/or SAP IDM Runtime. All communication takes place from NextGen to SAP NetWeaver via HTTPS exclusively, which helps to maintain both SAP IDM’s and NextGen’s strong security posture.
Unlike ad-hoc queries executed interactively, IDM CSV Server operations can be fully automated and require no human in the loop. This is particularly important if you need to migrate incrementally, e.g. company by company or application by application. Such scenarios may require operating SAP IDM and NextGen in parallel for a while, increasing the relevance of IDM CSV Server’s capability to automate and repeat the full data migration cycle as often as you need.
Migrate Wherever You Want
IDM CSV Server is distributed as a ZIP archive containing both server-side components to be deployed on SAP IDM and client-side components to be deployed on or near NextGen. As you may expect, running integrated into SAP IDM implies that IDM CSV Server’s server-side aims to be compatible with any database and operating system supported by SAP IDM. IDM CSV Server’s client-side, on the other hand, is a Linux shell script designed to be compatible with many Linux distributions.
“Wait, does that mean you only support NextGen running on Linux?”
Corporate IT departments in general and SAP customers in particular have an undeniable preference for Microsoft Windows. IAM vendors naturally tailor their products to this reality. So if your specific choice of NextGen makes it difficult or impractical to run Linux shell scripts, don’t worry – developing your own custom client for IDM CSV Server is pretty simple.
To make the task as easy as possible, IDM CSV Server comes with an HTTP request collection for Bruno, an open source REST API client. Bruno is a developer tool specifically designed for prototyping and testing REST APIs – very much like Postman. Using IDM CSV Server’s Bruno collection and the official Linux client as a blueprint, any IT professional – possibly equipped with some GenAI coding assistance – can build their own client in whatever language they prefer, be it PowerShell, Python, Node.js or others.
Export What You Need
“How do we make sure the CSV contains everything we need?”
Knowing which parts exactly of SAP IDM’s data must be migrated to NextGen requires in-depth understanding of your IAM scenario; there’s no single answer applicable to all. The good news is that IDM CSV Server includes plenty of content designed to make your conceptual task easier. More importantly, it does not require you to cram everything into one single CSV.
CSV Export Types
IDM CSV Server has a concept of so-called CSV Export Types, or export types for short. Each export type comes as an SAP IDM package, which can be deployed separately as needed. Export types help you organize your SAP IDM data into multiple CSV files properly, each with its own structure and purpose. The key elements of each export type are:
- unique name of export type
- database query for the input
- attribute mapping for the CSV output
IDM CSV Server includes a number of ready-made export types corresponding one-to-one to SAP IDM entry types, plus the special ASSIGNMENT export type:
| CSV Export Type | SAP IDM Entry Type |
| PERSON | MX_PERSON |
| ROLE | MX_ROLE |
| PRIVILEGE | MX_PRIVILEGE |
| ASSIGNMENT | n/a |
Hence, you just need to pick and choose to have all standard attributes of several standard SAP IDM entry types at your disposal with zero effort. However, it’s important to understand that export types are not limited to SAP IDM entry types. Any result set from any SQL query can be an export type, making IDM CSV Server as flexible as traditional “database sync” options.
Customization
If your SAP IDM schema contains custom attributes that you wish to migrate, all you need to do is add these custom attributes to IDM CSV Server’s existing export types. If you have custom entry types as well, creating dedicated new export types for them is equally straight-forward. You’ll need SAP IDM Developer Studio to copy packages and modify job passes, but nothing more – no custom coding in JavaScript or Java is required.
The following diagram illustrates the process of creating a custom export type -a one-time development activity- and then generating and downloading the resulting CSV data at runtime. The latter can be executed once or scheduled to run periodically in case you need to transfer updates from SAP IDM to NextGen repeatedly.
Reference Attributes Quirks
When evaluating IDM CSV Server, you’ll notice that out-of-the-box export types, such as PERSON, only contain regular attributes such as MX_FIRSTNAME and MX_LASTNAME, but no reference attributes. MXREF_MX_ROLE, MXREF_MX_PRIVILEGE and the like are all excluded from CSV, which might seem counter-intuitive at first. Knowing which identity is assigned to which roles and/or privileges is, after all, key to IAM’s core business. Everyone will need that information in NextGen, with no exceptions.
This is where the built-in export type ASSIGNMENT comes in. ASSIGNMENT is basically a one-to-one copy of SAP IDM’s database view IDMV_LINK_EXT, with all details like direct/indirect, assignment status and more included. Hence, IDM CSV Server by default suggests to migrate assignments in a format where each assignment is a single CSV row.
This is in sharp contrast to the alternative of including assignments directly in their respective entry type’s CSV, such as that of PERSON, and representing MXREF attributes as multi-value columns with single values separated by a separator character.
Having a dedicated export type ASSIGNMENT has two main benefits:
- Zero information loss: all assignment properties from SAP IDM remain at your disposal in NextGen
- Superior performance: requires constant memory and linear runtime for any data volume
As always, flexibility comes at a cost. Evaluating your migrated identities’ role assignments, for instance, requires that you import three CSV files (PERSON, ASSIGNMENT and ROLE) into NextGen and then perform a three way join
PERSON > ASSIGNMENT > ROLE
much in the same way as you would query the SAP IDM Database using an SQL join:
IDMV_ENTRY_SIMPLE > IDMV_LINK_EXT > IDMV_ENTRY_SIMPLE
With that said, there’s nothing that stops you from including reference attributes directly in your own export types or from adding them to IDM CSV Server’s out-of-the-box export types if you need to.
Each export type package comes with a switch FX_USE_FASTTRACK that you can toggle to switch from the default, SQL-based data retrieval to a more generic, identity store-based data retrieval. With the latter option, adding reference attributes is as simple as adding a new line to the load job’s attribute mapping. No changes to the database query are required.
Where to Go from Here?
If you’re an SAP IDM user and wondering whether IDM CSV Server could help you migrate to NextGen more easily, feel free to install and evaluate IDM CSV Server in your development system. Installing all parts of IDM CSV Server is extensively documented in the user manual, a must read for anyone considering to try out IDM CSV Server.
As with all software, IDM CSV Server contains bugs and I cannot rule out that you’ll run into one of them when using IDM CSV Server. Should this happen, please use the project’s bug tracker on Codeberg to report your issue.
Please do not use the comment section of this post for reporting bugs. SAP Community is a great place to discuss, but its definitely inadequate for tracking software issues and their resolutions.
What’s Next(Gen)?
This article has focused on the old world: “How do I get data out of SAP IDM?” Much more interesting, however, is the follow-up question: “How do I get it into NextGen?”
Answering the latter question will require a separate, product-specific discussion for each relevant migration target. Generic aliases like “NextGen” be gone, time to talk about specific IAM products available on the market today. Of course, no one knows every IAM vendor inside out. As for myself, I’ll hence limit myself to writing about one or two of them in the foreseeable future and cover those in depth, rather than merely skim the surface of many.
I suppose that you, my dear reader, might be an expert yourself in one of those IAM solutions that aim to succeed SAP IDM. It doesn’t matter whether your candidate is established or niche, on-prem or cloud, open source or proprietary. If you’re interested in publishing about data migration from SAP IDM to yours with the help of IDM CSV Server – definitely get in touch! I’d be delighted to contribute and collaborate.
Let me conclude this post on a personal and thankful note: Farewell, Data Synchronisation Engine. You’ve been fun to work with.
#abap