Imperfect PDF Digital Signatures

Introduction

Let's start with a demonstration. Here is a sample PDF/A-3a document, digitally signed with an eiDAS approved PAdES LTV signature (eiDAS is the EU digital signature project, and PAdES is the type of PDF digital signature it requires. We'll come back to both.)

We'll call this file file1.pdf.

That's how it looks in Acrobat Pro 2017.011.30144 for Mac, the latest version as of today. Note the metadata, the signature validity and - because it is a PDF/A-3a document - the logical structure, which makes the PDF suitable for use with screen readers and other assistive technology (you can hear the output in the video). There's nothing wrong with this file.

Here is the same file, with the same signature, after we've messed with it. We'll call this file2.pdf.

We've changed the Metadata and we've changed the Logical Structure (the internal structure optionally added to a PDF to make it easier to use with assistive technologies, such as screen readers). But Acrobat says the file is unchanged:

Document has not been modified since this signature was applied

If you're new to PDF digital signatures this is a very surprising result. Identifying changes is literally the point of a digital signature. Isn't it? It must be a bug.

Ceci n'est pas un bug

We found the metadata issue first, a couple of weeks ago, and sent a testcase to the PDF Association mailing list showing just that. The response from Adobe was that Acrobat has its own signature policy for documents modified after signing (paraphrased for brevity), and that the file as supplied was perfectly fine (verbatim).

Why? Because file2.pdf contains a revision to file1.pdf. The second revision alters the metadata and structure, but the original revision is still signed, unaltered and available to view by clicking on "Click to view this version". Revisions to signed files are not explicitly disallowed in PDF - in fact they're necessary to counter-sign a PDF, or to add long-term validity records to a PAdES signature (to record the validity of a signature at some point in time after signing).

But exactly what can be changed in a revision is not defined. So if you modify a PDF after signing, you have no way of knowing if it will invalidate the signature, or even - as shown above - if the change will be noted.

If you modify a PDF after signing, you have no way of knowing if it will invalidate the signature, or even if the change will be noted.

Adobe feel this situation is acceptable, but I do not. I'll try to explain why.

Some history

Digital signatures were first added to PDF in 2000. Back then there was no attempt to limit what could be done in a revision after signing - you could delete all the pages and replace them with new ones, and Acrobat would simply tell you the file had been changed after signing, and give you the option of viewing the signed version.

But there was never any doubt if a file had been changed. To the right is Acrobat 8 after verifying file3b.pdf. This is a modified version of file3.pdf (a valid PDF signed with the original type of PDF digital signature), with a single space added at the end of the file. You get the same message if you add a new page, as we did in file4.pdf. No matter how small, all modifications are reported the same way.

Revisions to a signed file weren't always desired, so in 2003 Adobe introduced Modification Detection and Prevention (MDP - also called Certifying signatures). Between PDF 1.5 and PDF 1.7 this algorithm detailed exactly what changes could be made to a PDF after signing, but the exact algorithm was dropped when PDF 1.7 became ISO32000, and replaced with a choice between either ... any changes shall invalidate the signature or ... permit modifications that are appropriate for form field or comment workflows.

So you had a choice. As of Acrobat 8, you could choose a regular digital signature and know that any revisions could be made to the file, and that any change, no matter how small, would be reported on. Or you could choose a Certifying digital signature and select between allowing no changes at all, or modifications that are appropriate for form field or comment workflows. While this is a bit vague, implementers had the algorithm from PDF 1.7 to use for reference.

This was effectively a signature policy - the rules you choose to apply when validating a signature. Adobe have never published their signature policy, but I suspect I've summed it up fairly closely in the above paragraph. That Adobe hadn't divulged it wasn't terribly important.

Signature Policy Changes

Things started to change around Acrobat 9. Our file4.pdf (the one where we added a page after signing) went from "valid but modified" in Acrobat 8 to "invalid" in Acrobat 9, while file5.pdf, where we changed only the metadata, went from "valid but modified" to "the document has not been modified".

I confess I didn't notice this change at the time - we'd supported digital signature for 6 years at this point, and didn't anticipate a breaking change from Adobe. No one asked us about it, so I guess none of our customers did either. When asked for detail on the changes, Adobe wouldn't give them: it's a proprietary business decision for our customers and (we) will not be publicly disclosing the details and our customers are well aware of this and (in many cases) are HAPPY about our policy – which is why it is there.

How can you be happy with a policy if you don't know what it is? I asked that too:

... if you want someone else to create or validate a signature that meets a policy, you have to tell them what the policy is

Very true. However, we have publicly stated (in this group and at ISO meetings) that we don’t expect others to match our policy in this regard.

In other words: Adobe have their policy, it's going to remain secret, and they do not expect other software to agree with Acrobat on whether a signature in a modified file is valid or not. Perhaps we can guess what their policy is? Open any of the signed documents above and bring up the signature properties dialog:

No, I didn't specify that. Acrobat has decided that on my behalf. But at least it gives us a hint of their policy: signing and commenting are allowed.

Here is file8.pdf - an example we worked up while testing. It has form fields and is signed. We then signed it a second time to give us file9.pdf and yes, both signatures are valid. Great!

But here is file10.pdf - signed and valid, exactly the same as file8.pdf, but with an additional radio button. We signed this a second time to give us file11.pdf but now the first signature is invalid. Not great. In fact, any revision to this file will invalidate the original signature: file12.pdf changes only the "last modified" date, but the signature is still invalidated. (It was fine in Acrobat 9 and X but became invalid in XI - it looks like that signature policy has seen quite a bit of revision.)

Test results

In total we tested 52 documents containing a valid, non-certifying digital signature that Acrobat considered valid. We counter-signed each, and found in 10 of those 52 documents, Acrobat now considered the first signature invalid. We inspected those files to confirm that the minimum possible set of changes was applied. Further testing on the 10 documents that failed showed that for 9 of them, we could also invalidate the original signature by simply adding a revision that changed the "last modified" date in the Metadata. We were unable to find a pattern as to why some documents could be counter-signed, and some could not.

For completeness, we also tested adding a revision in Acrobat, by adding signature verification information before re-saving the file. This also invalidated the signatures.

In 10 of 52 files, applying a second signature to a PDF resulted in the original signature becoming invalid. For 9 of those 10, simply revising the metadata was enough to invalidate the signature. Which files were invalidated was impossible to predict in advance.

I suspect if Adobe's customers are happy with this, it's because they've never used this workflow - probably because they've never tried to roll out a large-scale infrastructure project built on the technology. Enter eiDAS and PAdES.

Why is this a problem?

PAdES is a big deal. It's of one the underlying signature technologies for eiDAS, a massive, multi-national project woven into the legal framework of Europe. Digitally signed documents will be going back and forth between different parties with only minimal coordination, across market segments, between individuals, corporations and governments. Interoperability is fundamental to its success.

...we don’t expect others to match our policy in this regard

This statement amounts to "interoperability doesn't matter to us". And why should it? Adobe's turnover in 2018 was $9 billion, so they can be forgiven for standing on a few ants. Relatively speaking, everyone else in the PDF ecosystem is insignificant. What does it matter if their policy is proprietary?

Let me offer a viewpoint, as one of the ants.

1. Trust is lost.

   Without knowing the signature policy, you cannot be confident in the signature. If Alice signs a document, and Bob verifies it, neither can be certain the PDF hasn't been modified by Eve in such a way that Acrobat won't notify them of the change. We've demonstrated this is possible, right at the start. If Bob is reading with a screen reader, a non-certifying signature is essentially useless.

   Adobe point out that If the document hasn’t been modified, then you will find our policy is consistent with that of 32K and PAdES - in other words, if no changes are made after signing, then the validity is very tightly defined. True. But this is the equivalent of saying "our locks are fine so long as no-one tries to pick them".

   The more importance given to a digital signature, the more incentive there is to break it. eiDAS gives PDF signatures more importance than they've ever had. The signature must be tamper-resistant. Its validity after modification cannot be left undefined.

2. Interoperability across software is lost.
   With an all-Acrobat ecosystem, this wouldn't matter as much. But projects like eiDAS will not be built exclusively with Adobe's products. Signatures will be applied on mobile phones, notarised with Java applications and verified and processed by the government on mainframes. Some of these workflows will not feature Acrobat at all.
   
   In fact, for all my criticism of Adobe, publishing their signature policy isn't enough; a standard policy needs to be included in the specification, otherwise the multiple vendors in this space will be operating in the dark. Web-browser incompatibility is now a thing of the past because browser vendors chose to cooperate on specifications and test development. The PDF industry must do the same.
3. Interoperability across time is lost.
   PDF/A and PAdES LTV both demonstrate the use of PDF is an archival format. Signed documents and contracts may be kept for many years: if the signature was valid on signing, will it still be valid in 30 years? Without a published signature policy, no one can say. Even if you sign in Acrobat 2019 and validate in Acrobat 2049, who can say if the policy will be the same? It's already changed at least once. It may even change again soon. PAdES LTV describes in great detail a way to track changes to certificate validity over time. But without tracking changes to the signature policy itself, it can only ever be an incomplete solution.
4. Some operations are impossible.
   Prior to Acrobat 9, you could add a page to a signed document. Now you can't. There's no technical reason for this: it's simply a policy decision and one you, when you sign a document, have no say in. There are workflows that require this - for example when notarising a signed document, the only way to be sure nothing is obscured is to add the digital apostille on a new page.
5. Testing is impossible.
   Without a published policy, no-one knows if a signature in a modified document is valid by accident or by design. Am I supposed to be able to change the logical structure after signing or not? I have no idea. So it doesn't get tested, and bugs don't get found, and when they are it's by accident - there's no way to write a test plan, after all.
   
   And if you do stumble across a bug, whose bug is it? If our software validates a signature and Acrobat doesn't, who is right and who is wrong? Without a clear answer, work in that area is unable to progress. We make a note and move on; other bugs remain undiscovered, and the overall quality of the software is diminished.
6. Proprietary is not an acceptable word in ANY technology woven into legislation. Not ever. PDF has been an ISO specification since 2008, and PAdES signatures an ETSI specification since 2010. This stuff is not secret!

Management summary, mitigation and evidence

You're still with me, or you've skipped to the end. Either way, let me recap:

1. It is possible to make some changes to a signed document without Acrobat reporting on them.
2. Those same changes applied to a different document will invalidate the signature in Acrobat.
3. We don't know before we make a change, if it will invalidate a signature or not.
4. We don't know if this is by design or by accident. It may be a bug, or it may be policy.
5. This has been the case for some time, but it matters a lot more now eiDAS is here. Although it is not just PAdES signatures that are affected.
6. The uncertainty is hampering development and testing. It is damaging the PDF ecosystem, and damaging trust.
7. The only way this will change is if the exact method of determining the validity of signatures after revision becomes a public part of the specification.

Do we have any recommendations? Yes, yes we do.

• If you are applying a digital signature to a PDF, always apply a certifying signature. Specifically, one that disallows any changes at all. This will remove any ambiguity about what changes are allowed after signing: the answer is none. So this won't help if the document needs counter-signing later, and it also won't allow for for the later addition of long-term validation information. Acrobat disables the option to add LTV to documents signed with certifying signatures, and when we've added LTV information with our software, Acrobat invalidates the signature. By design or by accident? Of course, we don't know.
• One of the few things we can say for certain is that any changes to the page tree - ie. the addition or deletion of pages after signing - will invalidate the signature. This has been a constant since Acrobat 9, and I think we can safely assume that it is intentional, although it prevents some types of operation.
• If you are verifying a PDF in Acrobat, always choose "Click to view this version" - even if the dialog claims the PDF is unaltered since signing. It is the only way you can be sure what you are seeing is unaltered.

Finally, I need to explicitly state that we did report at least some of the above to Adobe, several weeks ago - specifically the metadata issue. Their response was essentially that there was nothing to see here. I beat on this drum for some time, making most of the points above, but got no further than having my opinions noted.

I am not attacking Adobe for having bugs in their software - let he who is without sin, etc. - I am attacking them for deliberately choosing an approach that means those bugs will be harder to find, that will intentionally damage interoperability, and that will damage the public perception of PDF digital signatures, which form a not insignificant part of our business.

Our customers are working on eiDAS-related projects. They are testing and asking "why can I not countersign this document without Acrobat invalidating it?". They're asking this a lot, they're absolutely right to, and I can't give them an answer. Although I suppose I just have.

You can download the source code we used to develop most of these tests here. You'll need a PKCS#11 keystore (as described here) for some, and a regular keystore for others, as well as our PDF Library of course.