ENI (4) ENI Document for Sedipualba: Resolutions in ENI format

 

1. Introduction

The purpose of this post is to get the Resolution Book in ENI Format. The steps to be followed are:

  • Download an Excel document with all the SEGRAS's id 
  • Filter the SEGRAs that are from the year and that have created a resolution.
  • For each SEGRA :
  • Use the SEGRA Web Service function Segra.ObtenerPropuestaInfoV2 to obtain information
  • Open the URL: https://TOWN.sede.dival.es/firma/infocsv.aspx?csv="+csv and parse some information (SignatureURL, SignerIds (idFirmantes), signers(firmantes) and signing data (fecha)
  • If the resolution is not included in an "Expediente" then include it to an "Expediente" of "Resolutions without Expediente" using the WS function   Segex.AñadirDocSegraAExpedientePorId
  • Now as an Expediente is known, a call to this WS function Segex.ObtenerInfoDocumentoConCsv gives us much more information (pkdocumento, some ENI information, ..) 

  • If we want to obtain all the information about constructing the hash on which the signature is made, just parse the URLs in SignatureURL (that was parsed in a previous step) and add "&it=true" to the URL, and obtain textoFirmante in UTF-8, the salt and the hash calculated by Sedipualba. If you want to verify the hash you must download the "Original document" too, but this document is needed for constructing the ENI document, so it will be downloaded next.
  • Save all the information about a resolution in a file using JSON format
  • Download the "original document" from the URL https://TOWN.sede.dival.es/firma/documento.aspx?tipo=original&csv="+ csv + "&modo=abrir"
  • Optionally you can download the "authentic document" from the URL   https://TOWN.sede.dival.es/firma/documento.aspx?csv="+ csv + "&modo=abrir"
  • Download the signatures in XADES-A format from the URL that is constructed by replacing in the SignatureURL these strings "infofirmante.aspx?" by "documento.aspx?tipo=xades&" and also replacing "&" by "&"

  • Now we have information enough for building the resolutions in ENI format.
Take into account using JAXB for parsing XML and do not use Jackson as it fails!

2. Consuming WS from Sedipualba

See the previous post about consuming Web Services. Here is the class used for consuming the needed WS calls (WSSedipualba.java)

package openadmin.sedipualba.decrets;
import java.util.Properties;

import com.fasterxml.jackson.databind.ObjectMapper;

import ximodante.basic.utils.basic.ExecutionTypeEnum;
import ximodante.basic.utils.basic.PropertyUtilsEdu;
import ximodante.basic.utils.ws.WsOutput;
import ximodante.ws.alba.WsAlba;

public class WSSedipualba {
    
    //Get Information about a SEGRA by its idSegra
    public static String SegraObtenerPropuestaInfoV2(int pkPropuesta) throws Exception  {
        Properties propsWs=PropertyUtilsEdu.getProperties(ExecutionTypeEnum.NO_JAR, "ws_encrypt");
        String[] appKeyOpt= {"alba","directorio","prod"};
        long timeOut=20000L;
        WsAlba wsAlba = new WsAlba(propsWs, appKeyOpt,true, timeOut);
        String output=null;
        //System.out.println("call_01_01-------------------------------");
        try {
            WsOutput wsOutput=wsAlba.ObtenerPropuestaInfoV2(pkPropuesta);
            System.out.println("Resultado: OK="+wsOutput.isOK() + " Mensaje:" + wsOutput.getMessage());
            Object result=wsOutput.getOutput();
            output=new ObjectMapper().writeValueAsString(result);
            System.out.println("Resultado="+output);
                
        } catch (Exception e) {e.printStackTrace();}    
        return output;
        
    }
    
    //Get Information about a SEGRA by its idSegra
    public static String SegexObtenerInfoDocumentoConCsv(String csv, String expedient) throws Exception  {
        Properties propsWs=PropertyUtilsEdu.getProperties(ExecutionTypeEnum.NO_JAR, "ws_encrypt");
        String[] appKeyOpt= {"alba","directorio","prod"};
        long timeOut=20000L;
        WsAlba wsAlba = new WsAlba(propsWs, appKeyOpt,true, timeOut);
        String output=null;
        //System.out.println("call_01_01-------------------------------");
        try {
            WsOutput wsOutput=wsAlba.ObtenerInfoDocumentoConCsv(csv,expedient);
            System.out.println("Resultado: OK="+wsOutput.isOK() + " Mensaje:" + wsOutput.getMessage());
            Object result=wsOutput.getOutput();
            output=new ObjectMapper().writeValueAsString(result);
            System.out.println("Resultado="+output);
                
        } catch (Exception e) {e.printStackTrace();}    
        return output;
            
    }
    //AñadirDocSegraAExpedientePorId
    public static String SegexAñadirDocSegraAExpedientePorId(int idSegra, String expedient, String descripcion, int pkCarpeta, boolean accesoInteresados) throws Exception  {
        Properties propsWs=PropertyUtilsEdu.getProperties(ExecutionTypeEnum.NO_JAR, "ws_encrypt");
        String[] appKeyOpt= {"alba","directorio","prod"};
        long timeOut=20000L;
        WsAlba wsAlba = new WsAlba(propsWs, appKeyOpt,true, timeOut);
        String output=null;
        //System.out.println("call_01_01-------------------------------");
        try {
            WsOutput wsOutput=wsAlba.AñadirDocSegraAExpedientePorId(expedient, idSegra, descripcion, pkCarpeta, accesoInteresados);
            System.out.println("Resultado: OK="+wsOutput.isOK() + " Mensaje:" + wsOutput.getMessage());
            Object result=wsOutput.getOutput();
            output=new ObjectMapper().writeValueAsString(result);
            System.out.println("Resultado="+output);
                
        } catch (Exception e) {e.printStackTrace();}    
        return output;
            
    }
        
}

3. Download the Excel file of the SEGRAS and select the resolutions of one year

Enter in SEGRA, select "Administrar"


Then press "Listar Todos" button

And the download is done.

Have a look at the file and filter the fields whose first column begins with "Resolución Nº" and contains the desired year (in my case 2022)


Now you can see the resolution number and the URL to get the information

And here is the java code to get the values resolution number and id Segra from the excel file
//==================BEGIN CODE============================================
/**
     * Get the url of the SEGRA and an array List of Decret number and id Segra ordered by decret number
     * @param fileName
     * @return
     * @throws IOException
     * @throws OpenXML4JException
     * @throws SAXException
     * @throws ParserConfigurationException
     */
    //public static List<int[]>getIdsSegra(String excelFileName, int year) throws IOException, OpenXML4JException, SAXException, ParserConfigurationException {
    public static Object[]getIdsSegra(String excelFileName, int year) throws IOException, OpenXML4JException, SAXException, ParserConfigurationException {
        var aObj=new Object[2];
        var lstIds =new ArrayList<int[]>();
        String url="";
        int[] ii= {1};
        //For discarding the resolutions of a different year and segras without a resolution
        //Predicate<List<String>> myPredicate=(e -> true);
        Predicate<List<String>> myPredicate=(e -> 
            (StringUtils.indexOf(e.get(0), "Resolución Nº") == 0) &&
            (StringUtils.indexOf(e.get(0), "/"+year) >=22) && 
            (StringUtils.indexOf(e.get(0), "/"+ year) <=30)
            //(e.get(0).
        );
        List<String> lst=new ArrayList<String>();
        int i=0;
        for (Object mylst: ExcelUtilsXML.getData(excelFileName,ii,myPredicate,false)) {
            lst=(List<String>)mylst;
            String numDecret=arrngDecretNum(lst.get(0));
            String idSegra=arrngSegraNum(lst.get(6));
            int[] s= {Integer.parseInt(numDecret),Integer.parseInt(idSegra)};
            lstIds.add(s);
            if (i==0) url=lst.get(6);
            i++;
        }
        
        //Ordering
        Comparator<? super int[]> intComp= (int[] a, int[] b) -> Integer.compare(a[0], b[0]);
        lstIds.sort(intComp);
        aObj[0]=url;
        aObj[1]=lstIds;
        return aObj;
    }
//==================END CODE============================================

4. The whole process (SedipualbaDecretsUtilsBo.java)

This is the code
package openadmin.sedipualba.decrets;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.function.Predicate;

import javax.xml.parsers.ParserConfigurationException;

import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.text.StringEscapeUtils;
import org.apache.poi.openxml4j.exceptions.OpenXML4JException;
import org.json.JSONObject;
import org.xml.sax.SAXException;

import eni.doc.TipoDocumento;
import jakarta.xml.bind.JAXBContext;
import jakarta.xml.bind.JAXBException;
import jakarta.xml.bind.Marshaller;
import openadmin.utils.decrets.CessarLlibreDecretsENIv2Paso1;
import ximodante.basic.utils.basic.ExecutionTypeEnum;
import ximodante.basic.utils.office.excel.ExcelUtilsXML;

public class SedipualbaDecretsUtilsBo {
    
    //XML Marshaller for TipoDocumento
    private static Marshaller getENIDocMarshaller() throws JAXBException { 
        JAXBContext context = JAXBContext.newInstance(TipoDocumento.class);
        Marshaller mar= context.createMarshaller();
        mar.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
        return mar;
    }
    
    //Write a TipoDocumento object to a xml file
    private static void marshallENIDoc(TipoDocumento eniDoc, String fileName) throws JAXBException { 
        Marshaller mar=getENIDocMarshaller();
        mar.marshal(eniDoc, new File(fileName));
    }
    
    //Easily get arrays of strings
    public static String[] asArray(String... values) {
        return values;
    }
    
    /**
     * Arrange this string
     *   "Resolución Nº4136 de 30/12/2022: 013220073.003 DE.alçar.obj.compte.just.pag.a.just.3.2022.encar.mifsud.estruch"
     * to this one "4136"  that is the resolution's number
     * @param fila0
     * @return
     */
    private static String arrngDecretNum(String fila0) {
        int i="Resolución Nº".length();
        int j=fila0.indexOf(" de ");
        String a=StringUtils.substring(fila0, i,j).trim();
        return a;
    }
    
    /**
     * Arrange this string
     *    https://myayto.sede.dival.es/sefycu/detalle_docufirma.aspx?app=segra&id=644265
     * to this one "644265"  that is the Segra's number   
     * @param fila0
     * @return
     */
    private static String arrngSegraNum(String fila6) {
        String toFind="segra&id=";
        int i=StringUtils.indexOf(fila6, toFind) + toFind.length();   
        String a=StringUtils.substring(fila6, i).trim();
        return a;
    }    
    
    /**
     * Transforms the baseURL removing from the pattern and adding substitution
     * Example 
     *       baseUrl    = "https://myayto.sede.dival.es/sefycu/detalle_docufirma.aspx?app=segra&id=665840"
     *       fromString = "/sefycu"
     *       replacement= "/firma/infocsv.aspx?csv=KYMLDSDSDESDS"
     *       
     *       return       "https://myayto.sede.dival.es/firma/infocsv.aspx?csv=KYMLDSDSDESDS"
     *  
     * @param baseUrl The url to change
     * @param fromString  string that marks the possition to remove
     * @param replacement repacement string
     * @return
     */
    private static String arrngBaseUrl(String baseUrl, String fromString, String replacement) {
        int from=StringUtils.indexOf(baseUrl, fromString);
        String url=StringUtils.substring(baseUrl, 0, from)+replacement;
        return url;
    }    
    
    /**
     * Get the url of the SEGRA and an array List of Decret number and id Segra ordered by decret number
     * @param fileName
     * @return
     * @throws IOException
     * @throws OpenXML4JException
     * @throws SAXException
     * @throws ParserConfigurationException
     */
    //public static List<int[]>getIdsSegra(String excelFileName, int year) throws IOException, OpenXML4JException, SAXException, ParserConfigurationException {
    public static Object[]getIdsSegra(String excelFileName, int year) throws IOException, OpenXML4JException, SAXException, ParserConfigurationException {
        var aObj=new Object[2];
        var lstIds =new ArrayList<int[]>();
        String url="";
        int[] ii= {1};
        //For discarding the resolutions of a different year and segras without a resolution
        //Predicate<List<String>> myPredicate=(e -> true);
        Predicate<List<String>> myPredicate=(e -> 
            (StringUtils.indexOf(e.get(0), "Resolución Nº") == 0) &&
            (StringUtils.indexOf(e.get(0), "/"+year) >=22) && 
            (StringUtils.indexOf(e.get(0), "/"+ year) <=30)
            //(e.get(0).
        );
        List<String> lst=new ArrayList<String>();
        int i=0;
        for (Object mylst: ExcelUtilsXML.getData(excelFileName,ii,myPredicate,false)) {
            lst=(List<String>)mylst;
            String numDecret=arrngDecretNum(lst.get(0));
            String idSegra=arrngSegraNum(lst.get(6));
            int[] s= {Integer.parseInt(numDecret),Integer.parseInt(idSegra)};
            lstIds.add(s);
            if (i==0) url=lst.get(6);
            i++;
        }
        
        //Ordering
        Comparator<? super int[]> intComp= (int[] a, int[] b) -> Integer.compare(a[0], b[0]);
        lstIds.sort(intComp);
        aObj[0]=url;
        aObj[1]=lstIds;
        return aObj;
    }
    
    /**
     * Read the url="https://myayto.sede.dival.es/firma/infocsv.aspx?csv="+csv;
     * and extracts idDoc, signature URL, signer and data
     * @param csv
     * @return
     * @throws IOException
     */
    public static String[] getIdDoc_firmURL_firmant_data(String baseUrl, String csv) throws IOException {
        //3.1.2 Get pkdocumento and idFirmanteURL
        String str1=arrngBaseUrl(baseUrl,"/sefycu", "/firma/infofirmante.aspx?idFirmante");
        String[] filters= asArray(
            "<span id=\"ctl00_ctl00_cphM_cph_txtId\" class=\"campo campo150\">", //id documento
            //"href=\"https://myayto.sede.dival.es/firma/infofirmante.aspx?idFirmante", //url firmante
            "href=\""+str1, ///url firmante
            "style=\"white-space: pre-wrap;\">",    //descripcion firmante
            "_txtFecha\" class=\"campo-tabla\">" //fecha de firma
        );
    
        String[] from = asArray(
            "campo150\">",    
            "href=\"",
            "pre-wrap;\">",
            "campo-tabla\">"
        );
    
        String[] to = asArray(
            "</span>",
            "\"><img id=",
            "</span>",
            "</span>"
        );
        String url=arrngBaseUrl(baseUrl,"/sefycu", "/firma/infocsv.aspx?csv="+csv);
        //String url="https://mayayto.sede.dival.es/firma/infocsv.aspx?csv="+csv;
        //            https://mayayto.sede.dival.es/sefycu/detalle_docufirma.aspx?app=segra&id=665840        
        return WebPageDownloader.getAStringFromURL(url,filters,from,to);
    }
    
    /**
     * Complete the JSON object of information of a resolution
     * @param aSegra
     * @param csv
     * @param expedient
     * @param idDoc_firmURL_firmant_data
     * @return
     * @throws Exception
     */
    public static String getSegraJSON(String aSegra, String csv, String expedient, String[] idDoc_firmURL_firmant_data) throws Exception {
        String infoDoc=WSSedipualba.SegexObtenerInfoDocumentoConCsv(csv,expedient);
        JSONObject json=new JSONObject(infoDoc);
        //String descripcionAlternativa=json.get("descripcion").toString();
        String tipo=json.get("tipo").toString();
        String tipoUbicacion=json.get("tipoUbicacion").toString();
        String idTipoEni=json.get("idTipoEni").toString();
        String idEstadoElaboracion=json.get("idEstadoElaboracion").toString();
        String origenAdministracion=json.get("origenAdministracion").toString();
        String fecha=json.get("fecha").toString();
        String extension=json.get("extension").toString();
        String idSefycu=json.get("idSefycu").toString();
        String pkdocumento=json.get("pkdocumento").toString();
        String pkcarpetaPadre=json.get("pkcarpetaPadre").toString();
        
        String[] urls=StringUtils.split(idDoc_firmURL_firmant_data[1], "|");
        String[] firmantes=StringUtils.split(idDoc_firmURL_firmant_data[2], "|");
        String[] fechas=StringUtils.split(idDoc_firmURL_firmant_data[3], "|");
        
        //String kk=StringUtils.replace(urls[0],      "infofirmante.aspx", "documento.aspx?tipo=tdb&");
        String titol            =WebPageDownloader.downloadFromURL(StringUtils.replace(urls[0],      "infofirmante.aspx?", "documento.aspx?tipo=tdb&"));
        String infoFirmas="";
        for (int iiii=1;iiii<=urls.length;iiii++) {
            String[] filters= asArray( 
                    "<span id=\"ctl00_ctl00_cphM_cph_txtTextoFirmante\" style=\"white-space: pre;\">", //texto Firmante
                    "<span id=\"ctl00_ctl00_cphM_cph_txtSal", //salt
                    "<span id=\"ctl00_ctl00_cphM_cph_txtHuella"    //huella
                );
            
            String[] from = asArray( 
                    "pre;\">",    
                    ">",
                    ">"
                );
            
            String[] to = asArray(
                    "</span>",
                    "</span>",
                    "</span>"
                );
            
            String myUrl=urls[iiii-1]+"&it=true";
            String[] texto_salt_hash=WebPageDownloader.getAStringFromURL(myUrl, filters, from, to);
            
            infoFirmas+=
                    ",\"url"+iiii+"\":\"" +urls[iiii-1]+ "\"" +
                    ",\"firmante"+iiii+"\":\"" +StringEscapeUtils.escapeJson(firmantes[iiii-1])+ "\"" +
                    ",\"fecha"+iiii+"\":\"" +fechas[iiii-1]+ "\"" + 
                    
                    ",\"textoFirmante"+iiii+"\":\"" +StringEscapeUtils.escapeJson(texto_salt_hash[0])+ "\"" + 
                    
                    //",\"salt"+iiii+"\":\"" +StringEscapeUtils.escapeJson(salt)+ "\"" +
                    //",\"saltB64_"+iiii+"\":\"" +salt+ "\"" +
                    ",\"saltB64_"+iiii+"\":\"" +texto_salt_hash[1]+ "\"" +
                    
                    //",\"hash"+iiii+"\":\"" +StringEscapeUtils.escapeJson(hash)+ "\"" 
                    //",\"hashB64_"+iiii+"\":\"" +hash+ "\""
                    ",\"hashB64_"+iiii+"\":\"" +texto_salt_hash[2]+ "\""
                    ;    
            
        }
        
        aSegra=aSegra.substring(0, aSegra.length()-1) + 
            ",\"idDocumento\":"+idDoc_firmURL_firmant_data[0] +",\"expediente\":\"" + expedient+"\"" +
            //",\"descripcionAlternativa\":\"" +descripcionAlternativa +"\"" +
            ",\"tipo\":\"" +tipo +"\"" +
            ",\"tipoUbicacion\":\"" +tipoUbicacion +"\"" +
            ",\"idTipoEni\":\"" +idTipoEni +"\"" +
            ",\"idEstadoElaboracion\":\"" +idEstadoElaboracion +"\"" +
            ",\"origenAdministracion\":\"" +origenAdministracion +"\"" +
            ",\"fecha\":" +fecha  +
            ",\"extension\":\"" +extension +"\"" +
            ",\"idSefycu\":" +idSefycu +
            ",\"pkdocumento\":" +pkdocumento +
            ",\"pkcarpetaPadre\":" +pkcarpetaPadre +
            ",\"titol\":\"" +StringEscapeUtils.escapeJson(titol) +"\"" +
            infoFirmas +
               "}";
        
        return aSegra;
    }
    
    
    
    
    public static void execute(String[] args, ExecutionTypeEnum execType) throws Exception{
        int year=Integer.parseInt(args[1].trim());
        String organoDIR3=args[2].trim();
        String outputFolder=args[3].trim()+year;
        String decretExcelFileName=outputFolder+"/"+args[4].trim();
        String decretNumformat=args[5].trim();
        String defaultExpedient=args[6].trim();
        
        //1. File to save info of resolutions
        BufferedWriter out = new BufferedWriter(new FileWriter(outputFolder+"/INFO"+year+".txt"));
        
        //2. Read the resolution names with the the Segra's ids
        var objSegras= getIdsSegra(decretExcelFileName, year);
        String baseUrl=(String) objSegras[0];
        List<int[]>segras=(List<int[]>) objSegras[1];
        
        //3. For every resolution
        for (int i=0; i<segras.size(); i++) {    
            // 4. Call WS for getting resolution csv
            int[] decAndSeg=segras.get(i);
            int decNumber=decAndSeg[0];
            int segraId=decAndSeg[1];
            String aSegra=WSSedipualba.SegraObtenerPropuestaInfoV2(segraId);
                    
            //5. Get csv            
            JSONObject json=new JSONObject(aSegra);
            String csv=json.get("csvDecreto").toString();
            String descripcion=json.get("descripcion").toString();
            
            //6. Parse a URL for getting idDoc, URL of signature, signer and data 
            String[] idDoc_firmURL_firmant_data=getIdDoc_firmURL_firmant_data(baseUrl,csv); 
            
            //7. Get the resolution's expedient by parsing a URL
            String url=json.get("urlPublicaDetalle").toString();
            String expedient=WebPageDownloader.getStringFromURL(
                url,
                "Target=\"_blank\">Expediente", 
                ">Expediente ",
                ":");
            
            //8. If the resolution has no expedient then the "expedient de decrets sense expedient" is assigned
            if (expedient==null || expedient.length()==0) {
                expedient=defaultExpedient;
                String infoExp=WSSedipualba.SegexAñadirDocSegraAExpedientePorId(segraId, expedient, descripcion, -1, false);
            }
            //8. If the resolution is in several expedients, the first is got
            expedient=StringUtils.split(expedient, "|")[0];
                        
            //9. Complete JSON object with info of a Resolution
            aSegra=getSegraJSON(aSegra, csv, expedient, idDoc_firmURL_firmant_data);
            
            //10. Save to a file
            out.write(aSegra + "\n");
            out.flush();
            
            //11. Download Original document //url="https://myayto.sede.dival.es/firma/documento.aspx?tipo=original&csv="+ csv + "&modo=abrir";
            url=arrngBaseUrl(baseUrl,"/sefycu","/firma/documento.aspx?tipo=original&csv="+ csv + "&modo=abrir");
            InputStream in = new URL(url).openStream();
            String decOrig=outputFolder+"/Decret_"+year+"_"+String.format(decretNumformat, decNumber)+".orig.pdf";
            Files.copy(in, Paths.get(decOrig), StandardCopyOption.REPLACE_EXISTING);
            byte[] docContent = FileUtils.readFileToByteArray(new File(decOrig));
            
            //12. Download authentic //url="https://myayto.sede.dival.es/firma/documento.aspx?csv="+ csv + "&modo=abrir";
            url=arrngBaseUrl(baseUrl,"/sefycu","/firma/documento.aspx?csv="+ csv + "&modo=abrir");
            in = new URL(url).openStream();
            String decAuth=outputFolder+"/Decret_"+year+"_"+String.format(decretNumformat, decNumber)+".aut.pdf";
            Files.copy(in, Paths.get(decAuth), StandardCopyOption.REPLACE_EXISTING);
           
            //13. Download Signatures
            int iii=1;
            String[] urls=StringUtils.split(idDoc_firmURL_firmant_data[1], "|");
            List<byte []> firmaContent=new ArrayList<byte[]>();
            for (String myUrl: urls) {
                url=StringUtils.replace(myUrl, "infofirmante.aspx?", "documento.aspx?tipo=xades&");
                url=StringUtils.replace(url, "&amp;", "&");
            
                //Download the signature
                in = new URL(url).openStream();
                String decSign=outputFolder+"/Decret_"+year+"_"+String.format(decretNumformat, decNumber)+".firm_" + iii++ +".xades.xml";
                Files.copy(in, Paths.get(decSign), StandardCopyOption.REPLACE_EXISTING);
                byte[] frmContent = FileUtils.readFileToByteArray(new File(decSign));
                firmaContent.add(frmContent);
            }
            //14. Get Doc in ENI format
            TipoDocumento eniDoc=ENIDocSedipualba.getENIDocumento(year, decNumber, docContent, firmaContent, aSegra, organoDIR3, decretNumformat);
            String docEniJaxbFName=outputFolder+"/"+eniDoc.getContenido().getId()+".xml";
            marshallENIDoc(eniDoc,docEniJaxbFName);
        }
        out.close();
    }

    public static void main(String[] args) {
        
        //OJO: Hen de tindre creat l'expedient "1234567K" que arreplega els decrets sense expedient
        try { //                                           descripcio        any    organoDIR3  carpeta a guardar    "fitxer excel a llegir format num decret" "expedient"
            SedipualbaDecretsUtilsBo.execute(new String[]{"LLIBRE_DECRETS","2022","L01462384","/home/eduard/SEGRA","SEGRAAdministrador.xlsx","%05d",         "1234567K"   }, ExecutionTypeEnum.NO_JAR);
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }

}


5. The utility class WebPageDownloader.java

This utility class is for parsing information from a URL
package openadmin.sedipualba.decrets;

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.CharBuffer;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.HashMap;
import java.util.Map;

import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.text.StringEscapeUtils;


    


public class WebPageDownloader {
    
    /**
     * Read the html of a webpage and extracts a string (in this case another url)
     * 
     * @param url (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN"
     * @param filterString The string to contain (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN")
     * @param from The string not included that is the mark of the beginning of the string to pick
     * @param to The string not included that is the mark of the end of the string to pick
     * @return
     * @throws Exception 
     * @throws IOException
     */
    public static void printHTMLFromURL(String url) throws Exception  {
        
        
        byte[] fileContent = IOUtils.toByteArray(new URL(url));
        
        String output=new String(fileContent,StandardCharsets.UTF_8);
        System.out.println(output);
    }
    
    /**
     * Read the html of a webpage and extracts a string (in this case another url)
     * 
     * @param url (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN"
     * @param filterString The string to contain (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN")
     * @param from The string not included that is the mark of the beginning of the string to pick
     * @param to The string not included that is the mark of the end of the string to pick
     * @return
     * @throws Exception 
     * @throws IOException
     */
    public static String downloadFromURL(String url) throws Exception  {
        
        byte[] fileContent = IOUtils.toByteArray(new URL(url));
        
        String output=new String(fileContent,StandardCharsets.UTF_8);
        return output;
    }
    
    public static byte[] downloadBytesFromURL(String url) throws Exception  {
        
        byte[] fileContent = IOUtils.toByteArray(new URL(url));
        
        return fileContent;
    }
    

    /**
     * Read the html of a webpage and extracts a string (in this case another url)
     * 
     * @param url (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN"
     * @param filterString The string to contain (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN")
     * @param from The string not included that is the mark of the beginning of the string to pick
     * @param to The string not included that is the mark of the end of the string to pick
     * @return
     * @throws IOException
     */
    public static String getStringFromURL(String url, String filterString, String from, String to) throws IOException {
        // Translate address of the webpage to the URL form
        
        String output="";
        URL address = new URL(url);
        // Instantiate connection object
        HttpURLConnection connection = (HttpURLConnection) address.openConnection();

        // Set the properties of the connection (optional step)
        connection.setRequestMethod("GET"); // GET or POST HTTP method, GET is default
        connection.setRequestProperty("accept-language", "en"); // add headers to the request

        // Proceed with downloading only if server sends code OK 200
        int http_status = connection.getResponseCode();
        if (http_status != HttpURLConnection.HTTP_OK) {
            throw new IOException("Server response with error, code: " + http_status);
        }

        // Set up input stream to read data from the server
        //BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream(), StandardCharsets.UTF_8));

        // Set up output stream to the file on disk
        BufferedWriter out = new BufferedWriter(new FileWriter("/home/eduard/kk/page.html"));
        //BufferedWriter out1 = new BufferedWriter(new FileWriter("/home/eduard/kk/page_ok.html"));
        // Save data to the html file
        String line;
        String sep="";
        while ((line = in.readLine()) != null) {
            out.write(line + "\n");
            if(filterString !=null && StringUtils.contains(line, filterString)) {
                String myURL="";
                int i=line.indexOf(from) + from.length();
                int j=line.indexOf(to);
                if (j<0) {
                    while(j<0) {
                        myURL+=StringUtils.substring(line, i)+"\n";
                        line = in.readLine();
                        out.write(line + "\n");
                        i=0;
                        j=line.indexOf(to);
                    }
                    myURL+=StringUtils.substring(line, i, j);
                    
                } else myURL+=StringUtils.substring(line, i, j);
                output+= sep + myURL;
                sep="|";
                myURL=StringEscapeUtils.unescapeHtml4(myURL);
            }
        }    
        // Close input output streams after work is done
        in.close();
        out.close();
        //out1.close();
        return output;
    }
    
    /**
     * Read the html of a webpage and extracts a string (in this case another url)
     * 
     * @param url (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN"
     * @param filterString The string to contain (for instance https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN")
     * @param from The string not included that is the mark of the beginning of the string to pick
     * @param to The string not included that is the mark of the end of the string to pick
     * @return
     * @throws IOException
     */
    public static String[] getAStringFromURL(String url, String[] filterString, String[] from, String[] to) throws IOException {
        // Translate address of the webpage to the URL form
        
        String[] output=new String[filterString.length];
        URL address = new URL(url);
        // Instantiate connection object
        HttpURLConnection connection = (HttpURLConnection) address.openConnection();

        // Set the properties of the connection (optional step)
        connection.setRequestMethod("GET"); // GET or POST HTTP method, GET is default
        connection.setRequestProperty("accept-language", "en"); // add headers to the request

        // Proceed with downloading only if server sends code OK 200
        int http_status = connection.getResponseCode();
        if (http_status != HttpURLConnection.HTTP_OK) {
            throw new IOException("Server response with error, code: " + http_status);
        }

        // Set up input stream to read data from the server
        //BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream(), StandardCharsets.UTF_8));
        
        // Set up output stream to the file on disk
        BufferedWriter out = new BufferedWriter(new FileWriter("/home/eduard/kk/page.html"));
        //BufferedWriter out1 = new BufferedWriter(new FileWriter("/home/eduard/kk/page_ok.html"));
        // Save data to the html file
        String line;
        String sep="";
        while ((line = in.readLine()) != null) {
            out.write(line + "\n");
            for(int ii=0; ii<filterString.length; ii++) {
                
                if(filterString[ii] !=null && StringUtils.contains(line, filterString[ii])) {
                    int i=line.indexOf(from[ii]) + from[ii].length();
                    int j=line.indexOf(to[ii]);
                    String myData="";
                    if (j<0) {
                        while(j<0) {
                            myData+=StringUtils.substring(line, i) + "\n";
                            line = in.readLine();
                            out.write(line + "\n");
                            i=0;
                            j=line.indexOf(to[ii]);
                        }
                        myData+=StringUtils.substring(line, i, j);
                        
                    } else myData+=StringUtils.substring(line, i, j);
                    myData=StringEscapeUtils.unescapeHtml4(myData);
                    
                    //myData+=StringUtils.substring(line, i, j);
                    if (output[ii]==null || output[ii].length()==0) {
                        sep="";
                        output[ii]="";
                    }
                    else sep="|";        
                    
                    output[ii]+= sep + myData;
                    
                }    
            }
        }    
        // Close input output streams after work is done
        in.close();
        out.close();
        //out1.close();
        return output;
    }
        
    public static void main(String[] args) throws Exception {
        /*
        String myStr=getStringFromURL(
                    "https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN",
                    "href=\"https://tavernesdevalldigna.sede.dival.es/firma/infofirmante.aspx?idFirmante", 
                    "href=\"",
                    "\"><img id=");
        
        System.out.println(myStr);
        */
        printHTMLFromURL("https://tavernesdevalldigna.sede.dival.es/firma/infofirmante.aspx?idFirmante=8139043&csv=PMAARFTY39HF49A2FYZ4&it=true");
        //System.out.println(Base64.getEncoder().encodeToString(downloadBytesFromURL("https://tavernesdevalldigna.sede.dival.es/firma/documento.aspx?tipo=sal&idFirmante=8139043&csv=PMAARFTY39HF49A2FYZ4")));
        //System.out.println(Base64.getEncoder().encodeToString(downloadBytesFromURL("https://tavernesdevalldigna.sede.dival.es/firma/documento.aspx?tipo=dgh&idFirmante=8139043&csv=PMAARFTY39HF49A2FYZ4")));
        //System.out.println(downloadFromURL("https://tavernesdevalldigna.sede.dival.es/firma/documento.aspx?tipo=dgh&idFirmante=8139043&csv=PMAARFTY39HF49A2FYZ4"));
        //https://tavernesdevalldigna.sede.dival.es/firma/documento.aspx?tipo=dgh&idFirmante=8139043&csv=PMAARFTY39HF49A2FYZ4
        /*
        //<span id="ctl00_ctl00_cphM_cph_txtId" class="campo campo150">
        String myStr=getStringFromURL(
                "https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARKZ4C49CEJPNYQHP",
                "<span id=\"ctl00_ctl00_cphM_cph_txtId\" class=\"campo campo150\">", 
                "campo150\">",
                "</span>");
    
        System.out.println(myStr);
        
        myStr=getStringFromURL(
                "https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN",
                "<span id=\"ctl00_ctl00_cphM_cph_txtId\" class=\"campo campo150\">", 
                "campo150\">",
                "</span>");
    
        System.out.println(myStr);
        
        String[] filters= 
            {    "<span id=\"ctl00_ctl00_cphM_cph_txtId\" class=\"campo campo150\">",
                "href=\"https://tavernesdevalldigna.sede.dival.es/firma/infofirmante.aspx?idFirmante"
            };
        
        String[] from =
            {    "campo150\">",    
                "href=\""
            };
        
        String[] to =
            {    "</span>",
                "\"><img id="
                    
            };
        
        
        String[] AStr=getAStringFromURL(
                "https://tavernesdevalldigna.sede.dival.es/firma/infocsv.aspx?csv=PMAARFNQHXN9PYZ447YN",
                filters,from,to);
                
    
        for (String ss: AStr) System.out.println(ss);
        
        
        //printHTMLFromURL("https://tavernesdevalldigna.sede.dival.es/sefycu/detalle_docufirma.aspx?app=segra&id=516511&codigo=s90atT8kOpN8nQd");
        String myStr=getStringFromURL(
                "https://tavernesdevalldigna.sede.dival.es/sefycu/detalle_docufirma.aspx?app=segra&id=516511&codigo=s90atT8kOpN8nQd",
                "Target=\"_blank\">Expediente", 
                ">Expediente ",
                ":");
    
      System.out.println(myStr);
      */
    }    
        
        
}

6. The utility class ENIDocSedipualba.java

This utility class is for managing ENI documents

package openadmin.sedipualba.decrets;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Date;
import java.util.GregorianCalendar;
import java.util.List;

import javax.xml.datatype.DatatypeConfigurationException;
import javax.xml.datatype.DatatypeFactory;
import javax.xml.datatype.XMLGregorianCalendar;

import org.json.JSONObject;

import eni.doc.EnumeracionEstadoElaboracion;
import eni.doc.Firmas;
import eni.doc.SignatureType;
import eni.doc.TipoContenido;
import eni.doc.TipoDocumental;
import eni.doc.TipoDocumento;
import eni.doc.TipoEstadoElaboracion;
import eni.doc.TipoFirmasElectronicas;
import eni.doc.TipoFirmasElectronicas.ContenidoFirma;
import eni.doc.TipoFirmasElectronicas.ContenidoFirma.CSV;
import eni.doc.TipoFirmasElectronicas.ContenidoFirma.FirmaConCertificado;
import eni.doc.TipoMetadatos;

import jakarta.xml.bind.JAXBContext;
import jakarta.xml.bind.JAXBElement;
import jakarta.xml.bind.JAXBException;
import jakarta.xml.bind.Unmarshaller;
/*
import openadmin.eni.documento.TipoFirmasElectronicasEdu;
import openadmin.eni.documento.TipoFirmasElectronicasEdu.ContenidoFirma;
import openadmin.eni.documento.TipoFirmasElectronicasEdu.ContenidoFirma.CSV;
import openadmin.eni.documento.TipoFirmasElectronicasEdu.ContenidoFirma.FirmaConCertificado;
*/
import openadmin.utils.jackson.IJacksonEdu;


public class ENIDocSedipualba {
    
    public static String CSV_REGULATION="BOP Albacete 111-17-P-3";
    //private static String FORMAT_NUM_DECRET="%05d";
    
    private static Unmarshaller getSignatureUnmarshaler() throws JAXBException { 
        JAXBContext context = JAXBContext.newInstance(SignatureType.class);
        Unmarshaller mar= context.createUnmarshaller();
        return mar;
    }
    
    private static SignatureType getSignatureFromByteArray(byte[] signatureData) throws JAXBException { 
        Unmarshaller mar=getSignatureUnmarshaler();
        JAXBElement<SignatureType> jaxb= (JAXBElement<SignatureType>) mar.unmarshal(new ByteArrayInputStream(signatureData));
        SignatureType mySignature=(SignatureType) jaxb.getValue();
        return mySignature;
    }

        
    
    public static List<String>getORGANO(String organoDIR3) {
        List<String>lstOrg=new ArrayList<String>();
        lstOrg.add(organoDIR3);
        return lstOrg;
    }
    
    /*
    public static String getNomDoc(int any, int pkDoc, String codi4Xifres, String desc, String organoDIR3) {
        //ES_L01462384_2022_DOC_11504451_0000_Decretinicimod
        String idEspecific=("DOC_"+ pkDoc+"_"+ codi4Xifres +"_"+ desc).substring(0, 30);
        return "ES_"+organoDIR3+"_"+any+"_" + idEspecific; 
        
    }
    */
    
    public static String getIdentificadorDoc(int any, int pkDoc, int numDecret, String organoDIR3, String decretNumformat) {
        return "ES_" + organoDIR3 + "_" + any + "_DOC_RESOLUCIO_" + String.format(decretNumformat, numDecret) + "_"+ pkDoc;
    }
    
    public static XMLGregorianCalendar getFecha(String longFecha) {
        XMLGregorianCalendar xmlGregCalFecha=null;
        Date date = new Date(Long.parseLong(longFecha));
        try {
            //xmlGregCalFechaCaptura = DatatypeFactory.newInstance().newXMLGregorianCalendar(doc.getFechaAlta().toString());
            GregorianCalendar gc = new GregorianCalendar();
            gc.setTime(date);
            xmlGregCalFecha =DatatypeFactory.newInstance().newXMLGregorianCalendar(gc);
            
        } catch (DatatypeConfigurationException e) {
            e.printStackTrace();
        }
        return xmlGregCalFecha;
    }
    
    /**
     * 1. ENI: Contenido
     * Convert from "ObjectEduType" contenido to "TipoContenido" from ENI
     * @param contenido
     * @return
     */
    //public static TipoContenido getENIContenido(ObjectEduType xsigContenido) {
    public static TipoContenido getENIContenido(int any, int numDecret, byte[] docContent, String extension, String pkDoc, String organoDIR3, String decretNumformat) {
        TipoContenido contenido=new TipoContenido();
        
        try {
            //FileUtilsEdu.writeToFile("/home/eduard/Baixades/0prova4a.pdf", docContent);
            contenido.setValorBinario(new String(Base64.getEncoder().encode(docContent)));
            //FileUtilsEdu.writeToFile("/home/eduard/Baixades/0prova4b.b64", contenido.getValorBinario());
            //FileUtilsEdu.writeToFile("/home/eduard/Baixades/0prova4c.pdf", Base64.getDecoder().decode(contenido.getValorBinario()));
            //contenido.setId("Decret_"+ any + "_"+ String.format("%04d",numDecret));
            contenido.setId(getIdentificadorDoc(any, Integer.parseInt(pkDoc), numDecret, organoDIR3, decretNumformat));
            //contenido.setNombreFormato(doc.getNomFormato().toLowerCase());
            contenido.setNombreFormato(extension);
            
            
            return contenido;
            
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
    
    
    public static TipoMetadatos getENIMetadatos(String lineaSegra, int year, String organoDIR3, int numDecret, String decretNumformat ) {
        JSONObject json=new JSONObject(lineaSegra);
        String fechaDecreto=json.get("fechaDecreto").toString();
        String pkDoc=json.get("pkdocumento").toString();
        String desc=json.get("descripcion").toString();
        String eelab=json.get("idEstadoElaboracion").toString();
        String origenAdministracion=json.get("origenAdministracion").toString();
        String idTipoEni=json.get("idTipoEni").toString();
        
        
        TipoMetadatos eniMetadatos=new TipoMetadatos();
        
        eniMetadatos.setVersionNTI("http://administracionelectronica.gob.es/ENI/XSD/v1.0/documento-e");
        //eniMetadatos.setIdentificador(getNomDoc(year, Integer.parseInt(pkDoc), "0000", desc, organoDIR3));
        eniMetadatos.setIdentificador(getIdentificadorDoc(year, Integer.parseInt(pkDoc), numDecret, organoDIR3, decretNumformat));
        
        TipoEstadoElaboracion tEE=new TipoEstadoElaboracion();
        //tEE.setValorEstadoElaboracion(EnumeracionEstadoElaboracion.EE_01); //Original
        tEE.setValorEstadoElaboracion(EnumeracionEstadoElaboracion.fromValue(eelab));
        eniMetadatos.setEstadoElaboracion(tEE);
        
        eniMetadatos.setOrgano(getORGANO(organoDIR3));
        eniMetadatos.setOrigenCiudadanoAdministracion(Boolean.valueOf(origenAdministracion));
        eniMetadatos.setFechaCaptura(getFecha(fechaDecreto));
        //eniMetadatos.setTipoDocumental(TipoDocumental.fromValue("TD01")); // Resolución
        eniMetadatos.setTipoDocumental(TipoDocumental.fromValue(idTipoEni)); // Resolución
        
        return eniMetadatos;
    }
    
    
    
    //public static Firmas getENIFirmas(String lineaSegra, int year, String organoDIR3, IJacksonEdu xmlMapper, String csv, String pkDoc, List<byte[]>  firmasXades) throws Exception {
    //public static Firmas getENIFirmas(String lineaSegra, int year, String organoDIR3, String csv, String pkDoc, List<byte[]>  firmasXades) throws Exception {
    public static Firmas getENIFirmas(int year, String organoDIR3, String csv, String pkDoc, List<byte[]>  firmasXades) throws Exception {
        //JSONObject json=new JSONObject(lineaSegra);
        //String fechaDecreto=json.get("fechaDecreto").toString();
        
        
        Firmas firmas=new Firmas();
        
        //List<TipoFirmasElectronicasEdu> lstFirmas=new ArrayList<TipoFirmasElectronicasEdu>();
        List<TipoFirmasElectronicas> lstFirmas=new ArrayList<TipoFirmasElectronicas>();
        
        //Add CSV signature
        //TipoFirmasElectronicasEdu tfrmCSV= new TipoFirmasElectronicasEdu();
        TipoFirmasElectronicas tfrmCSV= new TipoFirmasElectronicas();
        tfrmCSV.setTipoFirma("TF01"); //CSV
        CSV myCSV=new CSV();
        myCSV.setValorCSV(csv);
        myCSV.setRegulacionGeneracionCSV(CSV_REGULATION);
        tfrmCSV.setContenidoFirma(new ContenidoFirma());
        tfrmCSV.getContenidoFirma().setCSV(myCSV);
        lstFirmas.add(tfrmCSV);
        
        //Add XADES signatures
        for (int i=0; i<firmasXades.size(); i++) {
               //SignatureEduType mySignature=xmlMapper.ByteArrayToObject(firmasXades.get(i), SignatureEduType.class);
            //SignatureType mySignature=xmlMapper.ByteArrayToObject(firmasXades.get(i), SignatureType.class);
            SignatureType mySignature=getSignatureFromByteArray(firmasXades.get(i));
            
            
            //TipoFirmasElectronicasEdu tfrm= new TipoFirmasElectronicasEdu();
               TipoFirmasElectronicas tfrm= new TipoFirmasElectronicas();
            tfrm.setTipoFirma("TF03"); //Xades Enveloped
            tfrm.setContenidoFirma(new ContenidoFirma());
            tfrm.getContenidoFirma().setFirmaConCertificado(new FirmaConCertificado());
            tfrm.getContenidoFirma().getFirmaConCertificado().setSignature(mySignature);
            lstFirmas.add(tfrm);
        }
        firmas.setFirma(lstFirmas);
        return firmas;
    }    
    
    //public static TipoDocumento getENIDocumento(int any, int numDecret, byte[] docContent, List<byte[]> firmaContent, IJacksonEdu xmlMapper, String lineaSegra, String organoDIR3) throws Exception {
    public static TipoDocumento getENIDocumento(int any, int numDecret, byte[] docContent, List<byte[]> firmaContent, String lineaSegra, String organoDIR3, String decretNumformat) throws Exception {
        TipoDocumento eniDoc=new TipoDocumento();
        JSONObject json=new JSONObject(lineaSegra);
        String extension=json.get("extension").toString();
        String csv=json.get("csvDecreto").toString();
        String pkDoc=json.get("pkdocumento").toString();
        
        //1. Contenido
        eniDoc.setContenido(getENIContenido(any, numDecret, docContent, extension, pkDoc, organoDIR3, decretNumformat));
        
        //2.Metadatos
        eniDoc.setMetadatos(getENIMetadatos(lineaSegra, any, organoDIR3, numDecret, decretNumformat));
        
        //3. Firmas
        //eniDoc.setFirmas(getENIFirmas(lineaSegra, any, organoDIR3, xmlMapper,  csv, pkDoc,  firmaContent));
        //eniDoc.setFirmas(getENIFirmas(lineaSegra, any, organoDIR3, csv, pkDoc,  firmaContent));
        eniDoc.setFirmas(getENIFirmas(any, organoDIR3, csv, pkDoc,  firmaContent));
        
        return eniDoc;
    }
}



Comments

Popular posts from this blog

ORVE WS (Dynamic) (4) Jackson XML mapper

ENI (1) ENI Document OR the Spanish Electronic Administration Mafia

ORVE WS (Dynamic) (12) Defiining the control tables