Transcript
Hello. Issuing invoices is really important for business. That's how you get paid for the work that you have done. Usually an invoice contains an invoice name, the invoice date, the due date for the payment, the information about the receiver, the sender, the information about each line item for the invoice, the totals, and the information of how the payment should be done. And a problem with digital invoices is that they usually send in PDF format and it'll be hard to read the, the relevant information, uh, from the PDF format of invoices, especially when you have a lot of invoices that, uh, your company needs to pay. And usually there is a lot of tools that would help you read the, this information from invoices and solve a problem. For example, there is mind D where you can, uh, upload your PDF documents to them and they would, uh, extract the relevant information most often for invoices. That's their main business, uh, but they charge quite a lot. So it can be like 10 cents per page. And if you have a lot of, uh, small invoices to pay, they can get really expensive. There are also tools like, uh, AI tools like this invoice, uh, AI scanner or you can use, uh, JGPT like tools where you can upload an, uh, invoice and you've get the, some kind of, uh, j with the invoice information. There are also different tri tools like PDF reader that would basically just copy paste all the, uh, all the texts from an invoice into a string. But you would still need to extract the data into an object, uh, in a meaningful way where you can, the integration of the line items where you can get the information about the send the BAT send a name and so on. So this tool will not help you that much. Luckily, EU with its regulations want to solve this, uh, problem of, uh, written, uh, digital invoices. They have this e-invoicing, uh, project where they want to each PDF document that, uh, is an invoice to have an attached XML with all the, uh, relevant invoice data structured in, uh, standardized format. So your software will not need to read the PF document at all. The PDF will be just for the humans, and the software should read the attached XML that is present in the metadata. And the XML should have the, uh, information, uh, in the, a very structured format. So it should be the same for all, uh, invoices. And, uh, this has already been implemented, uh, in some countries, especially in France and Germany. So in France, uh, it has been implemented as as of 2026 for big companies, 2027 for small companies. So if you are a company that is, uh, issuing invoices to the French government, you will need to invoice, uh, to issue invoices that, uh, have this attached, uh, XML file with all the invoice metadata. So if you are building a invoicing software for the French or for the German market and you want to stay competitive in 2006, you will need to generate all your invoices that, uh, are valid in this factor x uh, uh, format. Talking about factor X format, that is how it, uh, like this invoices are called in France. In Germany they used to be called tuk third, but now Germany has uh, also started referring to them as factor X. And on this, uh, wonderful website, uh, you can see the logos of, uh, factor x, uh, compliant, uh, documents and you can download the technical specifications. So I downloaded the technical specifications and uh, I got, um, this folder. So first of all, I have the technical appendix. It's kind of a, uh, data schema of, uh, what the attached XML document can look like. Then, uh, there is, um, an Excel file with the example invoices and with the different types of data that can be present in an invoice. So yeah, like all of specifications, this is useful for you if you're building invoicing software. And, uh, here is, uh, actual great document that is, uh, quite readable that I read about, uh, what, uh, is the point of uh, having this e invoices, how they should work and so on. This is a good document. I recommend reading it if you are working on invoicing software. And it also has some examples of different, uh, PDF documents that are valid, uh, in this factor X format. So each of these documents will have an attached XML file, uh, on which you can train your software to read this, uh, factor X or invoice e invoice, uh, compliant PDFs. And that's kind of a tool that I built, uh, here. So I built a small tool, they can upload PDF documents and uh, if they contain, uh, this, uh, embedded XML in the right format, then uh, it'll, uh, extract the data. Let's have a look how it works here. I have some kind of invoice. I will, uh, upload this invoice into my app. And here I have, uh, an invoice object created. I have an the PDF invoice attached. Here it is. And uh, I have, uh, the information that it is valid in this factor X format and I read some invoice information. Let's compare this information. We have the invoice number. Here it is, we have the date, 11th, uh oh, 16th of November. Here it is. We have the you payable amount minus 233. Here it is. We have, uh, the sender name and VAT and the client name and, uh, VAT. Here it is. So my tool did not need to read the information, uh, like the text information in the PDF at all. It just found the attached uh, XML file and read this information directly from this XML file. And Here I will open the invoice. I extracted that the attached XML file. Let's have a look at how I do it in my Rails app itself. So, uh, let's start from the beginning. First of all, I have the invoices controller where I have this, uh, import action where I upload the, uh, file or multiple files. And from these files I create invoices with the attached, uh, PDF documents here and going to invoice.org beam, I have attached PDF documents. So an invoice will have attached PDF. And after creating the invoice object, I run this extract XML job and the extract XML job, they'll try to find an attached XML in this PDF. So let's have a look at the job itself. Here is the job. So, uh, as I read in the thing specification, uh, in this file, uh, there can be two valid file formats for the XML, either factor X or extract maybe for legacy Germany documents. And uh, I will, uh, basically take the PDF document content. I will open it with the gem HX PDFI installed this gem in my GEM file. The gel helps me extract the, the metadata from the PDF. So, uh, I run a few different manipulations to find the right document. And uh, if the document has the file name that is one of these, then I'm going to save the XML in a text field in my invoice. So in the schema invoice has, uh, text field XML document. And here is this XML document, uh, field. Here it is. I have extracted the XML. And you see in my app I have all these fields that get the information directly from the XML. I don't have each of them present, uh, in my invoice, uh, object. I just have the XML document from which I extract the invoice number, invoice date, and uh, so on. And, um, let's have a quick look at how this uh, works. So, uh, uh, let's say I'll go to the Rails console. I have the Rails console and I will try to use this, uh, gem hexa PDF to extract the XML from A PDF. So here, for example, have a PDF document in my fixtures, I will go by the file path. I will say file path equals this. And, uh, PDF equals hex, uh, pdf, DF, uh, re I think open. Let's have a look again. Uh, hexa pdf, DF document, document dot, open file path. So we have this PDF object and catalog equals PDF catalog. So catalog has, uh, names inside. It has embedded files, it has do value inside has names inside. It has, uh, this array with, uh, the name of the file and the file reference itself. Names do, uh, last or second, last, let's say. So it is, uh, reference object equals this rough. And we will say P, D, F. So PD, F, uh, object for this reference. And here we find this object attached to the PDF. And, uh, from this we'll find the, uh, let's say EF F, it's not so easy actually initially use the, uh, I think JIT or copilot to find this right reference to dig and find the right uh, x ml content itself. And uh, I will have to say stream, do stream. And here finally I have the PDF content itself. So you see XML version 1.0 and so on. So here's the PDF itself. So I needed do quite a lot of manipulations to get, uh, the PDF out of, uh, of X ml out of the PDF, uh, in the console. And I do it all in this PDF log to ML job. Also, I wrote a few tests. So, uh, test PDF blow to XML job. So I create an invoice, I attach, uh, A PDF, I find the, the attached XML or find the, uh, different fields in the xml. But another interesting thing, okay, we have, uh, attached X ml to the invoice. How do we read the invoice number, invoice date, and so on. There are actually a couple of ways. Here is one example of factor XXML concern. So I use X no XML to pass, uh, the XML field, the x ml document inside the invoice for different information like this would be how I would find the IB, this would be how I would find the, an account name and so on. But the better way, one that I like more is, uh, uh, searching for inside JSO. So here I use Knowledge Jam named Nori, and uh, I passed the XML document with Nori, um, and uh, it kind of turns it into JS O. And already with this Jason, I make some digging to find different fields. So I find the invoice date, invoice number, and so on, so on. And as this a concern I just added to the invoice model. And now invoice, uh, uh, invoice date would be passing in the attached XML file. So let's say the date will be, not this date and pause, but hello, let's refresh have we have this Hello instead. So yeah, this is basically how the app works. I've written some, uh, tests for the jobs for the invoices themselves, and the solution seems to be quite stable. So feel free to use it. It'll be present at Cogo slash factor X reader. And if you are a subscriber to res.com, then you will have access to this, uh, repository. And feel free to use this. And, uh, uh, yeah, it can be a seven stone for you to save some time and, uh, uh, pause invoices trying to find then attached XML and get the information from the XML instead of using different OCR like tools. So thank you for watching and see you in the next one.
2