Integration of serialization into key manufacturing processes

Today's post provides a current assessment of the integration of serialization into common pharma manufacturing processes.  Certainly not a new topic and one that has been covered in numerous articles from colleagues and vendors.  As a point of reference- nearly 18 months ago a great overview was provided by David Colombo of KPMG in Pharmaceutical Online (Read it here). 

How far have we come since then?   Did the industry heed the message or are we still treating serialization 'in a vacuum' and missing integration into key operations?

Let''s dive into two key manufacturing processes where ensuring proper serialization integration is critical.

Serial Number Reconciliation-  No pharma manufacturer (following GMP) would release product without quantity reconciliations being performed and documented in batch records.  Quantity of items packed, quantity of items sampled, quantity of items damaged/destroyed- this is the norm of any packaging operation.  So why should it be any different with serialization?    

With serialization an entirely new data set exists (serial numbers) which must align with the quantities documented in a batch record- So it stands to reason that checking the quantity of SNs against the quantities in batch records each and every time seems like a painfully obvious concept-  many will assume it's being done.    But if you are a manufacturer and have ever experienced mismatches in your serialization data then quite clearly reconciliation was missed somewhere along the way.  (Comment and let me know your experiences!!)

So, what's preventing serial number reconciliations from being a pervasive concept? In many cases it comes down to limitations of serialization solutions.  Many vendor solutions don't support the ability to do serial number reconciliations in two main areas:

• Packaging site/CMO solutions which only allow the collation and communication of serialization data to manufacturers at the time of shipment.    Following good practices- if a manufacturer only receives the serialization data for a batch after the product has been loaded onto a truck it is way too late in the process.  When designing packaging site/CMO integrations identify trigger points during the batch process which will support reconciliation (such as upon lot closure).    If either your CMO's system or your system can only support integration at time of shipment- be prepared to justify the risk of not being able to perform serial number-to-batch record reconciliation until 1) after you've had to provide batch release and 2) after your product is already in transit.  What's the preferred capability in this area? Some solutions are able to take in batch record inputs (even better if they are electronic batch records!!) and automatically compare against processed serialized data- providing alerts when quantities do not reconcile.  

• Solutions which do not support capture/communication of 'end-of-life' data (end-of-life covers any scenario where a serial number is no longer continuing in the commercial supply chain).  This has long been a heated debate within the industry with organizations deeply rooted on both sides of the fence- in regards to DSCSA some choose to follow the regulations and only capture data for the 'good' items that get shipped/sold while others will require tracking of all allocated serial numbers.  For what it's worth- I've always been firmly planted on the latter side.   I summarize the debate like this-  there is no right or wrong answer, but if you are only tracking the 'good' items then the extent of your organization's capability is serialization compliance, not true traceability.   Again, for many organizations there is nothing wrong with that as they're likely still 'checking the box', it just means additional areas will have to be addressed before being able to take advantage of other benefits of true traceability (e.g. brand protection, process/quality efficiency monitoring, etc.)

Serialized Shipments-  For many organizations, capturing serialized shipping information is still a future endeavor.   But as 2019 requirements (both regulatory and industry driven) start to surface the need to track items from manufacturing through distribution arises.   One practice, however, that appears time and again is the concept of 'Ship by Lot'.   In the old, quantity-based world this concept had merit as it was an easy way to update quantities of items in ERP systems.  In a serialized world, however, continuing this practice brings significant risk.   The principle is simple-  You don't ship lots, you ship shipments.   

What allows this practice to continue in a serialized world is that often a shipment equals a lot.  Again, in the old, quantity-based world it was easy to say "Ship Lot 123" and then enter the quantity actually being shipped.  In a serialized world, performing the same action "Ship Lot 123" makes a big assumption that all serial numbers which have been previously associated to that lot are now being shipped.   The problem is that organizations are not going through and actually scanning each serial number as part of an explicit shipping step and therefore systems have to 'infer' which items are now being shipped.   The serialized world relies on inference in situations where aggregations exist (physical product packed and sealed in larger physical containers) but shouldn't rely on inference for creating significant traceability steps (e.g. shipping).    

This practice is most commonly seen at companies who have chosen not to aggregate for the time being but are often 'forced' into generating a shipping step by either their vendor or a downstream customer.    Additionally, certain vendors have made this practice too accessible by making it a standard (and even recommended) capability in their platforms.

My summary:  Wait until you are aggregating before you start capturing serialized shipping information unless your volumes are so small you can warrant doing an explicit shipping step.   Moreover, once you are aggregating there is no reason to continue a practice like 'Ship by Lot'.   Instead perform a true shipping step which includes scanning of parent level containers.    

For companies which DSCSA is the first serialization regulation they are having to meet the key is to anticipate what life will be like once the regulations are actually live. Certainly, the focus needs to be on ensuring you are meeting the 'letter of the law', but don't overlook how properly integrating serialization into key manufacturing processes can help identify issues with your serialization data well in advance of it becoming a compliance item. Ultimately manufacturers are responsible for managing serialization within these common processes on a day-to-day basis- Don't let your vendor push you into practices that are easier for them to support, but introduce risk to your operations.

Needing to comply with EU FMD? Don't miss this critical GS1 Guidance on the proper use of NTINs in EPCIS

Recently I highlighted an important GS1 guidance which aimed to correct a common misuse of GLN/SGLNs in EPCIS events.  Next in our series is another GS1 guidance focusing on the proper encoding of NTINs in EPCIS events.   The full position paper can be found here.

As a general note, GS1 authors numerous position papers which are immensely helpful yet unfortunately often don't get the visibility/attention they deserve.   Check out the full set of position papers to ensure your implementations are in line with GS1 standards.

What is the root issue? NTINs (National Trade Identification Number) are product identifiers which are the result of country or market specific identifiers being incorporated into GS1's GTIN construct.  The NTIN concept is certainly not new but has seen increased significance given its broad use across Europe and thus its key role in ensuring compliance with EU FMD regulations.    

While there is a significant push to have companies transition from the use of NTINs to GTINs in any market where it is allowed, the purpose of GS1's latest position paper is to ensure in scenarios where NTINs must be used they are encoded correctly when populated in EPCIS events.  This is a critical concept as EPCIS is the de facto standard used to transfer serialization/traceability information between packaging sites/CMOs and MAHs- which in turn is the foundation for the regulatory required notifications to the EMVS (a.k.a EU Hub system).

While NTINs do inherit some attributes of its GTIN counterpart- such as ensured global uniqueness- NTINs have one significant difference in their makeup.  Unlike GTINs, NTINs do not incorporate a GS1 Company prefix.   Most serialization software solutions depend on the GS1 company prefix to properly convert GTINs from their 'human readable' format, such as you would see printed next to a barcode, to its equivalent (GTIN+SN) SGTIN 'URN' format which is required when populating in EPCIS.  Thus, NTINs not having an explicit GS1 company prefix makes it problematic for software solutions when then trying to encode the NTIN+SN into the standard SGTIN URN format.

What ambiguity does the position paper clear up?  Most importantly the position paper clearly states that NTINs are to be encoded as GS1 SGTINs when populated in EPCIS events and more specifically "an NTIN must be encoded as a 'one-off' GTIN with a GCP length of 12 digits".  The position paper also provides clear instruction as to how to convert an NTIN+SN into the SGTIN URN format.

Source: GS1- Guidance on using NTIN in EPCIS visibility events

How can I determine if my implementation is following GS1's guidance?  Watch for these key indicators that your implementation may NOT be meeting this GS1 guidance:

• Any reference or mention by your vendor to the concept of an 'SNTIN'.   SNTIN is a fictional term because, as the position paper notes, an NTIN in serialized form takes on the construct of a standard SGTIN.  
• EPC values which use prefixes such as 'urn:epc:id:ntin:...' or 'urn:epc:id:sntin:...' are not compliant.  GS1 does not sanction any prefix under its namespace that includes 'ntin' or 'sntin'.
• A lack of GS1 Company prefix configuration in your serialization solution.   Serialization solutions must allow for configuration of known or valid GS1 company prefixes in order to properly translate between an NTIN/GTIN +SN human readable format and its corresponding SGTIN URN format.  Without a GS1 Company prefix configuration solutions will force users to duplicate its GTIN/NTIN entry in both human readable and URN formats.  This adds significant overhead to master data management/configuration as well as an increased quality risk that the entered human readable and URN formats do not properly correlate.

When made aware of GS1's position paper this week one top pharma indicated (in regards to feedback provided from their serialization solution provider) that "everything we’ve been told previously goes against this."   This is concerning feedback as it indicates many companies are already down the path of mishandling NTINs encoded in their EPCIS integrations. 

As noted in prior posts, when non-compliant practices become widespread the greatest consequences ultimately fall back on the industry itself in the form of additional time/effort/cost.  By not enforcing Standards compliance, serialization solutions have to adapt to support both the compliant and non-compliant approaches- and this comes at the expense of the industry (e.g. customers) who end up having to pay for new integration 'maps' or platform 'enhancements' to the same solution providers who incorrectly handled NTINs in the first place.

As before the message to the industry is simple- increase the oversight of your implementations and leverage the wealth of readily-accessible, free content available to you. Leaving your implementations unchecked will result in highly customized and/or non-compliant partner integrations costing you more money in the long run.

Next phase of EPCIS hopes to bring standard in line with long proven technology concepts

Excited to see leading technology concepts being applied to EPCIS as part of GS1's EPCIS 2.0 call to action (Link)

JSON and REST have been leading formatting and integration approaches in IT applications across many industries for many years.

• JSON- compact data format/messaging that has shown significant benefits over XML, especially in high volume use cases which are paramount in serialization/track & trace applications

• REST- the overwhelming leading A2A/B2B integration approach, with significant benefits over asynchronous integration methods such as AS2 and the heavily outdated SOAP protocol.

It will be interesting to see how pharma vendors supporting EPCIS adapt to these potential changes.   Concepts like JSON and REST have been used in serialization/track&trace applications in other industries for many years.   Time for pharma to catch up, and make no mistake, this isnt about the industry looking for new and emerging technologies like AI and blockchain.  This is about the industry catching up to technology trends that were happening 5+ years ago. 

Take a look:

https://www.openfox.com/images/White_Papers/CPI_REST-SOAP_Paper.PDF

"Rather, the kinds of organizations that favor SOAP tend to be slower to change and more heavily driven by integrating with other government agencies that are behind in technology, or by grant programs tied to a particular technology stack, and they often have legacy systems that require the use of SOAP. "

http://www.pwc.com/us/en/technology-forecast/2012/issue2/features/feature-consumerization-apis.html

"Service-oriented architecture (SOA), which gained wide acceptance using web services built on SOAP, has been popular within organizations as a mechanism for sharing information across the enterprise. However, the use of a REST architecture, along with associated technologies such as JavaScript Object Notation (JSON), is accelerating the development and use of APIs. Some of the most popular services such as Twitter, Netflix, and Facebook are now processing API calls on the order of billions per day or month."

https://www.infoq.com/articles/rest-soap

Technology adoption, such as this, is fueled by the vendors whom the industry puts their trust in to provide the best solutions.  In pharma this certainly won't be seamless, however, as some large players have long ignored support for these concepts.    

You don't need to be an expert on these terms (leave that for the IT folks) but if your current implementation is heavily rooted in antiquated technologies like AS2, SOAP (and soon to be XML) or hasn't committed to a clear enhancement path as part of their roadmap you're well behind the curve.

Important Public Service Announcement from GS1 and what it means for your Serialization Implementation

GS1 recently released a PSA (Public Service Announcement) noting a specific practice which is currently widespread in pharma serialization implementations and is decidedly non-compliant with GS1's EPCIS standard.  In addition to being non-compliant the practice also infringes upon GS1's property.  See GS1's PSA at the end of this post.

Following GS1's lead I will help bring attention to these types of data compliance and quality issues through this blog.

What is the root issue? In simplistic terms many serialization vendors are communicating ’where' traceability events occur in a non-compliant fashion by populating the GS1 GLN (Global Location Number) incorrectly in EPCIS events.

How widespread is this issue?  Considering multiple 'top' L4/L5 serialization vendors support this non-compliant practice this is a highly widespread issue.  Numerous sources brought this to the attention of GS1 leading to the PSA.  Additionally, proper use of GLN/SGLN has been part of the EPCIS standard since its inception over 10 years ago- So this particular item is alarming not only because of its pervasiveness, but also because of how basic and longstanding the underlying concept is.

How does this impact me?  

• Non-compliant EPCIS, in any form, leads to challenges in sharing traceability data with downstream partners (Distribution, 3PL, Customers)
• Not addressing data quality items, such as this, immediately will end up costing the industry more time and money to correct these issues as the volume of live, serialization data increases (often paying the same vendors to fix their own mistakes)
• Widespread issues such as this introduces contractual concerns for those responsible for providing compliant EPCIS integrations
• Tolerating non-compliant data in your serialization implementation significantly reduces your ability to harness emerging technologies such as AI and blockchain which demand good quality data

How can I determine if my implementation suffers from this issue?  The key here is don't just take my word for it- you must check your data for yourself. 

I know for some that's easier-said-than-done but fortunately there is no lack of services and tools to quickly find standards compliance issues such as this:

• The late Ken Traub developed an immensely helpful, FREE tool which will instantly tell you whether an EPCIS message is compliant, as well as provide a summary breakdown of its content.  (Link)
• I have offered a FREE service to analyze serialization data and provide compliance feedback (Link)
• The Jennason Serialization Test Tool provides automated serialization test data generation which can be used to ensure your L4/L5 solution properly supports the standards and can also detect non-compliance (Link)

If the options above are still too much effort ask your L4/L5 provider if any of your data is non-compliant per this PSA.  Then let me know their response- I'm eager to hear!!

The message to the industry is simple- take responsibility for your implementations by leveraging the tools readily available to support you.  Otherwise the 6.6% compliance rate we saw with DSCSA barcodes will seem pretty good by comparison.

I appreciate the work being done by Ralph Troger (GS1 Germany) and Craig Alan Repec (GS1 Global) to educate the industry on the proper use of standards by highlighting these non-compliant practices at both a global and country level.  A second 'thank you' to GS1 Brasil for their efforts in making this PSA visible to their members. 

I look forward to GS1 US taking the same vigor in protecting their property and standards.

GS1 Public service announcement on proper use of GS1 EPC URIs

Lawsuit against HDA dropped.

Notice that Tracelink dropped its lawsuit against the HDA regarding the Origin service was released this week.  Read the press release here (Link).

The absolute correct outcome, in my opinion.   From the start this lawsuit misrepresented the scope of Origin and read more like a bloated piece of marketing material.  I think of no other term but ‘frivolous’ when I see it publicly stated this case was based on ‘misunderstandings’.

The practice of wholesalers/organizations setting the tone for track & trace adherence in the industry is nothing new.  If this practice caused such an anti-competitive environment for L4/L5 solution providers why didn’t we see lawsuits three years ago when T3 electronic exchange was ‘forced’ by some wholesalers even though not required by DSCSA?   The answer is because, at that time, providing electronic lot level T3s actually kept some of these same solution providers in business.   Seemed a little too convenient to start complaining about this practice now. 

My personal experience has been nothing but openness and collaboration with the HDA/ValueCentric teams who position the Origin service as a very complementary offering to L4/L5 solutions.   I look forward to my future efforts with them. 

The fact is the Origin service offers a very real and necessary function which is to guarantee the accuracy of product identification (GTINs) and related attributes.   We just saw how well DSCSA barcoding is progressing (Link) so services like Origin attempt to ensure we don’t repeat the same utter failure when it comes to DSCSA data.  Unfortunately, it only addresses one piece of the puzzle- product master data- so many data issues will still arise. Moreover, everyone should recognize Origin is not the only service that can meet this need.  Data pools, like GS1's GDSN, have for years been providing master data collaboration for numerous industries and across all segments (manufacturing, distribution, retail).  However, the reality is most pharma companies have not advanced to the point of adopting GDSN as part of a master data strategy and thus why services like Origin fill a specific and immediate need.

As for the press release, I’m encouraged by the mention of aligning with standards but unfortunately believe this is just marketing fluff once again- I’m happy to be proven wrong but don’t quite see the need for more workgroups/new standards when no less than two GS1 standards which support master data exchange have existed for 10+ years.  For me this, again, falls into the category of being a little too ‘convenient’- making public claims of alignment with GS1 standards for the marketing credit but then do little to actually deliver support in solutions.  It certainly wouldn’t be the first time this has happened- and continues to be pervasive- with numerous solution providers in this space.  

So, what does this all mean going forward?  For many this lawsuit was an interesting side-story to keep an eye on, but others may have some very real impacts and questions to be answered.  We now have one of the major track & trace providers who has undoubtedly caused many key wholesalers (who also operate many of the largest 3PLs) and their primary industry association to expend significant time and money to defend against this lawsuit. Do we really think that relationship is all warm-and-fuzzy?

Why is this important?  To this point there has been a practice for many pharmas to select L4/L5 vendors simply based off who their packaging provider/CMO recommended- but now the focus is rapidly shifting to integration with the same distribution/3PLs and wholesalers named in this lawsuit- all in support of the next wave of DSCSA milestones in 2019.  So now these pharmas, who took a short-term view into serialization compliance by focusing on packaging only, are left to wonder how the next phase of their serialization program will play out.  

A precursor of what's to come? Barcode issues are only the start...

GS1 US recently released a telling article about the current state of barcoding for DSCSA compliance (Link).  A great review and opinion from Dirk Rodgers was also released on Monday.  A highly suggested read (Link).

The crux of the study is that approx. 6.6% of all serialized barcodes evaluated by two of the largest wholesalers met expected quality requirements and DSCSA compliance.  The results of this barcoding survey, in my opinion, are staggering to say the least.   

Let's put this in perspective- imagine if the results had come back that 50% of the barcodes did not meet DSCSA compliance?   If these results held true, that means that in just 10 short months half of all products distributed in the US should be set aside for non-compliance.  That would be catastrophic to the industry.

So that fact that only 6.6% of barcodes evaluated met two of the largest wholesaler's expectations for quality and DSCSA compliance isnt just staggering- it should be a signal to the entire industry that the traditional ways of approaching serialization simply are not working. 

But what may be more interesting is-  How does the pharma industry view these survey results?   I'd love to think the majority of the industry is gravely concerned.  My fear is most are not.  My fear is many take a position of either:  A) I don't care,  B) That can't possibly apply to me because I use all of the 'leading' vendors or C) If that many are non-compliant then I'm probably in the same boat as everyone else and therefore don't need to worry about it.

It would generally be 'easy' to pin the blame for these results on CMOs/packaging sites- but doing so would miss the mark.  Non-compliance of this magnitude indicates a failure at multiple levels- this is an issue with both business and technical involvement, regulatory interpretation, requirements gathering, GS1 Standards education and, certainly, execution.    I have no doubt the majority of these implementations were fully tested/validated- which means these implementations were doomed for failure before they even started because they were being tested/validated against requirements which were already non-compliant.    

Part of this blame certainly goes back on vendors who are depended upon to be experts in this space.  Vendors at all levels should be able to recognize the issues that led to 6.6% compliance- that's not situational failure, that's systematic failure.  Even enterprise vendors, where most of my experience is rooted, are subject.  If your enterprise vendor didn't recognize these issues-  red flag.

But vendors can't be the only ones in the cross-hairs.  Whether the industry wants to admit it or not there has long been a mentality towards serialization to do as little as possible, as fast as possible for as cheap as possible.   This can't be ignored as a contributing factor.   A lack of necessary oversight and attention to detail has put the industry in the current situation and the scariest part might be- how many companies don't even know they have issues or are led to believe everything is 'OK'?

We now have a clear indicator of how well that mentality is working out for the industry.  And (sorry for the doom-and-gloom) my estimate is its only going to get worse when the focus shifts from barcoding to serialization data.   

The results from the GS1 survey are more likely to be a telling indicator rather than an unexpected blip on the industry's march towards serialization compliance. Where we go from here is in the hands of the industry-  the documentation, tools and services exist to help pharma companies get this 'right'.   Whether the industry chooses to pay attention and take the necessary steps remains to be seen.    

A free service to expose the truth about your serialization implementation

Drawing upon over 10 years of experience implementing almost every major serialization solution on the market today Jennason is offering a free evaluation of your serialization implementation- focusing on its adherence to GS1 standards and its ability to meet global regulations.

The reason for offering this service now is simple:  the industry is being significantly mislead through false claims and misinformation about the proper implementation of GS1 standards which are a necessity to make industry collaboration possible.  By offering a free service my hope is companies see no barrier in looking further into the problems which are besieging their implementation.

Reach out to Jennson to learn more about this free offering and to hear actual use cases of pharma companies who confronted these issues and took action to get their implementations back on track.