Merge branch 'dev' into feature/19-speed_test

This commit is contained in:
Nicolai Ort 2021-02-09 17:50:05 +01:00
commit 467d8df6db
5 changed files with 306 additions and 287 deletions

View File

@ -2,8 +2,20 @@
All notable changes to this project will be documented in this file. Dates are displayed in UTC. All notable changes to this project will be documented in this file. Dates are displayed in UTC.
#### [v0.1.3](https://git.odit.services/lfk/document-server/compare/v0.1.2...v0.1.3)
- 🚀Bumped version to v0.1.3 [`6a14232`](https://git.odit.services/lfk/document-server/commit/6a142328898d5b89fa11eaf033372971d1093b0c)
- 🧾New changelog file version [CI SKIP] [skip ci] [`b6296b8`](https://git.odit.services/lfk/document-server/commit/b6296b8d97cda943dfb5e11bc9dfbb2f363f5b81)
- Merge pull request 'Load more stuff from env feature/16-env_vars' (#17) from feature/16-env_vars into dev [`bc4d16e`](https://git.odit.services/lfk/document-server/commit/bc4d16e6f8959ed35d7e87647de84584cdfddd7b)
- Added new env vars to config [`3bb322e`](https://git.odit.services/lfk/document-server/commit/3bb322ede5db15a147c0d7a8db2a68ccb7fa2112)
- Added new env vars to readme [`b77bb3a`](https://git.odit.services/lfk/document-server/commit/b77bb3ad9dba9d73c2c81215ba57936192155a9a)
- Now loading interpolation vars from config/env [`b4ebae2`](https://git.odit.services/lfk/document-server/commit/b4ebae283b472b2f0c6e28caed49b30edb119585)
- 🧾New changelog file version [CI SKIP] [skip ci] [`a306009`](https://git.odit.services/lfk/document-server/commit/a30600943d01116b99e946cb705a16d0372b5095)
#### [v0.1.2](https://git.odit.services/lfk/document-server/compare/v0.1.1...v0.1.2) #### [v0.1.2](https://git.odit.services/lfk/document-server/compare/v0.1.1...v0.1.2)
> 7 February 2021
- Merge pull request 'Alpha Release 0.1.2 - Hotfix release' (#15) from dev into main [`123cf8a`](https://git.odit.services/lfk/document-server/commit/123cf8ad48a45fa10dcd5208215a6e525f31115a) - Merge pull request 'Alpha Release 0.1.2 - Hotfix release' (#15) from dev into main [`123cf8a`](https://git.odit.services/lfk/document-server/commit/123cf8ad48a45fa10dcd5208215a6e525f31115a)
- 🧾New changelog file version [CI SKIP] [skip ci] [`22b1e00`](https://git.odit.services/lfk/document-server/commit/22b1e0097efc865de9cc150cb0d0b99bf789b519) - 🧾New changelog file version [CI SKIP] [skip ci] [`22b1e00`](https://git.odit.services/lfk/document-server/commit/22b1e0097efc865de9cc150cb0d0b99bf789b519)
- 🚀Bumped version to v0.1.2 [`7e507d4`](https://git.odit.services/lfk/document-server/commit/7e507d4cc415877ac0b25503dc0ff9ecdceabf42) - 🚀Bumped version to v0.1.2 [`7e507d4`](https://git.odit.services/lfk/document-server/commit/7e507d4cc415877ac0b25503dc0ff9ecdceabf42)

151
README.md
View File

@ -1,75 +1,78 @@
# @lfk/document-server # @lfk/document-server
The document generation server responsible for creating pdfs for sponsoring contracts, certificates and more. The document generation server responsible for creating pdfs for sponsoring contracts, certificates and more.
This server doesn't interact with any database and can therefor be deployed on it's own. This server doesn't interact with any database and can therefor be deployed on it's own.
The basic generation mechanism makes the templates and routes interchangeable (if you want to expand or modify it). The basic generation mechanism makes the templates and routes interchangeable (if you want to expand or modify it).
## Quickstart 🐳 ## Quickstart 🐳
> Use this to run the document server in docker. > Use this to run the document server in docker.
1. Clone the repo or copy the docker-compose 1. Clone the repo or copy the docker-compose
2. Run in the folder that contains the docker-compose file: `docker-compose up -d` 2. Run in the folder that contains the docker-compose file: `docker-compose up -d`
3. Visit http://127.0.0.1:4010/docs to check if the server is running 3. Visit http://127.0.0.1:4010/docs to check if the server is running
## Dev Setup 🛠 ## Dev Setup 🛠
> Local dev setup > Local dev setup
1. Rename the .env.example file to .env (you can adjust app port and other settings, if needed) 1. Rename the .env.example file to .env (you can adjust app port and other settings, if needed)
2. Install Dependencies 2. Install Dependencies
```bash ```bash
yarn yarn
``` ```
3. Start the server 3. Start the server
```bash ```bash
yarn dev yarn dev
``` ```
## ENV Vars ## ENV Vars
> You can provide them via .env file or docker env vars. > You can provide them via .env file or docker env vars.
| Name | Type | Default | Description | Name | Type | Default | Description
| - | - | - | - | - | - | - | -
| APP_PORT | Number | 4010 | The port the backend server listens on. Is optional. | APP_PORT | Number | 4010 | The port the backend server listens on. Is optional.
| NODE_ENV | String | dev | The apps env - influences debug info. | NODE_ENV | String | dev | The apps env - influences debug info.
| EVENT_NAME | String | "Please set the event name" | The event's name - used to generate pdf text.
## Templates | CURRENCY_SYMBOL | String | "€" | The your currency's symbol - used to generate pdf text.
> The document server uses html templates to generate various pdf documents. | SPONSORING_RECEIPT_MINIMUM_AMOUNT | String | "10" | The mimimum total donation amount a sponsor has to donate to be able to receive a donation receipt - used to generate pdf text.
> The templates are stored in src/templates by default.
## Templates
We provide a set of default templates that we use for the ["Lauf für Kaya!" charity run](https://lauf-fuer-kaya.de). > The document server uses html templates to generate various pdf documents.
We use handlebars for templateing utilizing i18next for translation - the i18n string format in the templates is : `{{__ "string"}}` > The templates are stored in src/templates by default.
You can provide your own templates by replacing the ones we provided before compiling the project or by simply mounting your custom templates into the docker container.
We provide a set of default templates that we use for the ["Lauf für Kaya!" charity run](https://lauf-fuer-kaya.de).
The server currently needs the following templates to work: We use handlebars for templateing utilizing i18next for translation - the i18n string format in the templates is : `{{__ "string"}}`
* sponsoring_contract.html You can provide your own templates by replacing the ones we provided before compiling the project or by simply mounting your custom templates into the docker container.
### Sponsoring Contracts The server currently needs the following templates to work:
* sponsoring_contract.html
| Template Data | Type | Optional | Description
| - | - | - | - ### Sponsoring Contracts
| runners | array(Runner) | ❌ | The runner objects. We generate a contract for each runner on a new DIN-A5 page.
| Template Data | Type | Optional | Description
| - | - | - | -
## Recommended Editor | runners | array(Runner) | ❌ | The runner objects. We generate a contract for each runner on a new DIN-A5 page.
[Visual Studio Code](https://code.visualstudio.com/)
## Recommended Editor
### Recommended Extensions
[Visual Studio Code](https://code.visualstudio.com/)
* will be automatically recommended via ./vscode/extensions.json
* we also provide a config for i18n-ally in the .vscode folder ### Recommended Extensions
## Staging * will be automatically recommended via ./vscode/extensions.json
### Branches & Tags * we also provide a config for i18n-ally in the .vscode folder
* vX.Y.Z: Release tags created from the main branch
* The version numbers follow the semver standard ## Staging
* A new release tag automaticly triggers the release ci pipeline ### Branches & Tags
* main: Protected "release" branch * vX.Y.Z: Release tags created from the main branch
* The latest tag of the docker image get's build from this * The version numbers follow the semver standard
* New releases get created as tags from this * A new release tag automaticly triggers the release ci pipeline
* dev: Current dev branch for merging the different feature branches and bugfixes * main: Protected "release" branch
* The dev tag of the docker image get's build from this * The latest tag of the docker image get's build from this
* Only push minor changes to this branch! * New releases get created as tags from this
* To merge a feature branch into this please create a pull request * dev: Current dev branch for merging the different feature branches and bugfixes
* feature/xyz: Feature branches - nameing scheme: `feature/issueid-title` * The dev tag of the docker image get's build from this
* Only push minor changes to this branch!
* To merge a feature branch into this please create a pull request
* feature/xyz: Feature branches - nameing scheme: `feature/issueid-title`
* bugfix/xyz: Branches for bugfixes - nameing scheme:`bugfix/issueid-title` * bugfix/xyz: Branches for bugfixes - nameing scheme:`bugfix/issueid-title`

View File

@ -1,6 +1,6 @@
{ {
"name": "@odit/lfk-document-server", "name": "@odit/lfk-document-server",
"version": "0.1.2", "version": "0.1.3",
"description": "The document generation server for the LfK! runner system. This generates certificates, sponsoring aggreements and more", "description": "The document generation server for the LfK! runner system. This generates certificates, sponsoring aggreements and more",
"main": "src/app.ts", "main": "src/app.ts",
"scripts": { "scripts": {

View File

@ -1,198 +1,199 @@
import axios from 'axios'; import axios from 'axios';
import cheerio from "cheerio"; import cheerio from "cheerio";
import fs from "fs"; import fs from "fs";
import Handlebars from 'handlebars'; import Handlebars from 'handlebars';
import i18next from "i18next"; import i18next from "i18next";
import Backend from 'i18next-fs-backend'; import Backend from 'i18next-fs-backend';
import mime from "mime-types"; import mime from "mime-types";
import path from 'path'; import path from 'path';
import { PDFDocument } from 'pdf-lib'; import { PDFDocument } from 'pdf-lib';
import puppeteer from "puppeteer"; import puppeteer from "puppeteer";
import { Runner } from './models/Runner'; import { config } from './config';
import { RunnerGroup } from './models/RunnerGroup'; import { Runner } from './models/Runner';
/** import { RunnerGroup } from './models/RunnerGroup';
* This class is responsible for all things pdf creation. /**
* This uses the html templates from src/templates. * This class is responsible for all things pdf creation.
*/ * This uses the html templates from src/templates.
export class PdfCreator { */
private templateDir = path.join(__dirname, '/templates'); export class PdfCreator {
private browser; private templateDir = path.join(__dirname, '/templates');
private static interpolations = { eventname: "Lauf für Kaya! 2021", sponsoring_receipt_minimum_amount: '10', currency_symbol: "€" } private browser;
private static interpolations = { eventname: config.eventname, sponsoring_receipt_minimum_amount: config.sponsoring_receipt_minimum_amount, currency_symbol: config.currency_symbol }
/**
* Main constructor. /**
* Initializes i18n(ext), Handlebars and puppeteer. * Main constructor.
*/ * Initializes i18n(ext), Handlebars and puppeteer.
constructor() { */
this.init(); constructor() {
} this.init();
}
/**
* Main constructor. /**
* Initializes i18n(ext), Handlebars and puppeteer. * Main constructor.
*/ * Initializes i18n(ext), Handlebars and puppeteer.
public async init() { */
const minimal_args = [ public async init() {
'--autoplay-policy=user-gesture-required', const minimal_args = [
'--disable-background-networking', '--autoplay-policy=user-gesture-required',
'--disable-background-timer-throttling', '--disable-background-networking',
'--disable-backgrounding-occluded-windows', '--disable-background-timer-throttling',
'--disable-breakpad', '--disable-backgrounding-occluded-windows',
'--disable-client-side-phishing-detection', '--disable-breakpad',
'--disable-component-update', '--disable-client-side-phishing-detection',
'--disable-default-apps', '--disable-component-update',
'--disable-dev-shm-usage', '--disable-default-apps',
'--disable-domain-reliability', '--disable-dev-shm-usage',
'--disable-extensions', '--disable-domain-reliability',
'--disable-features=AudioServiceOutOfProcess', '--disable-extensions',
'--disable-hang-monitor', '--disable-features=AudioServiceOutOfProcess',
'--disable-ipc-flooding-protection', '--disable-hang-monitor',
'--disable-notifications', '--disable-ipc-flooding-protection',
'--disable-offer-store-unmasked-wallet-cards', '--disable-notifications',
'--disable-popup-blocking', '--disable-offer-store-unmasked-wallet-cards',
'--disable-print-preview', '--disable-popup-blocking',
'--disable-prompt-on-repost', '--disable-print-preview',
'--disable-renderer-backgrounding', '--disable-prompt-on-repost',
'--disable-speech-api', '--disable-renderer-backgrounding',
'--disable-sync', '--disable-speech-api',
'--hide-scrollbars', '--disable-sync',
'--ignore-gpu-blacklist', '--hide-scrollbars',
'--metrics-recording-only', '--ignore-gpu-blacklist',
'--mute-audio', '--metrics-recording-only',
'--no-default-browser-check', '--mute-audio',
'--no-first-run', '--no-default-browser-check',
'--no-pings', '--no-first-run',
'--no-zygote', '--no-pings',
'--password-store=basic', '--no-zygote',
'--use-gl=swiftshader', '--password-store=basic',
'--no-sandbox' '--use-gl=swiftshader',
]; '--no-sandbox'
];
await i18next
.use(Backend) await i18next
.init({ .use(Backend)
fallbackLng: 'en', .init({
lng: 'en', fallbackLng: 'en',
backend: { lng: 'en',
loadPath: path.join(__dirname, '/locales/{{lng}}.json') backend: {
} loadPath: path.join(__dirname, '/locales/{{lng}}.json')
}); }
await Handlebars.registerHelper('__', });
function (str) { await Handlebars.registerHelper('__',
return i18next.t(str, PdfCreator.interpolations).toString(); function (str) {
} return i18next.t(str, PdfCreator.interpolations).toString();
); }
this.browser = await puppeteer.launch({ headless: true, args: minimal_args }); );
} this.browser = await puppeteer.launch({ headless: true, args: minimal_args });
}
/**
* Generate sponsoring contract pdfs. /**
* @param runner The runner you want to generate the contracts for. * Generate sponsoring contract pdfs.
* @param locale The locale used for the contracts (default:en) * @param runner The runner you want to generate the contracts for.
*/ * @param locale The locale used for the contracts (default:en)
public async generateSponsoringContract(runners: Runner[], locale: string = "en"): Promise<Buffer> { */
if (runners.length == 1 && Object.keys(runners[0]).length == 0) { public async generateSponsoringContract(runners: Runner[], locale: string = "en"): Promise<Buffer> {
runners[0] = this.generateEmptyRunner(); if (runners.length == 1 && Object.keys(runners[0]).length == 0) {
} runners[0] = this.generateEmptyRunner();
if (runners.length > 50) { }
let pdf_promises = new Array<Promise<Buffer>>(); if (runners.length > 50) {
let i, j; let pdf_promises = new Array<Promise<Buffer>>();
for (i = 0, j = runners.length; i < j; i += 50) { let i, j;
let chunk = runners.slice(i, i + 50); for (i = 0, j = runners.length; i < j; i += 50) {
pdf_promises.push(this.generateSponsoringContract(chunk, locale)); let chunk = runners.slice(i, i + 50);
} pdf_promises.push(this.generateSponsoringContract(chunk, locale));
const pdfs = await Promise.all(pdf_promises); }
return await this.mergePdfs(pdfs); const pdfs = await Promise.all(pdf_promises);
} return await this.mergePdfs(pdfs);
await i18next.changeLanguage(locale); }
const template_source = fs.readFileSync(`${this.templateDir}/sponsoring_contract.html`, 'utf8'); await i18next.changeLanguage(locale);
const template = Handlebars.compile(template_source); const template_source = fs.readFileSync(`${this.templateDir}/sponsoring_contract.html`, 'utf8');
const result = template({ runners }) const template = Handlebars.compile(template_source);
const pdf = await this.renderPdf(result, { format: "A5", landscape: true }); const result = template({ runners })
return pdf const pdf = await this.renderPdf(result, { format: "A5", landscape: true });
} return pdf
}
/**
* Converts all images in html to base64. /**
* Works with image files in the template directory or images from urls. * Converts all images in html to base64.
* @param html The html string whoms images shall get replaced. * Works with image files in the template directory or images from urls.
*/ * @param html The html string whoms images shall get replaced.
public async imgToBase64(html): Promise<string> { */
const $ = cheerio.load(html) public async imgToBase64(html): Promise<string> {
$('img').each(async (index, element) => { const $ = cheerio.load(html)
let imgsrc = $(element).attr("src"); $('img').each(async (index, element) => {
const img_type = mime.lookup(imgsrc); let imgsrc = $(element).attr("src");
const img_type = mime.lookup(imgsrc);
if (!(img_type.includes("image"))) {
throw new Error("File is not image mime type"); if (!(img_type.includes("image"))) {
} throw new Error("File is not image mime type");
}
let image;
if (imgsrc.startsWith("http")) { let image;
image = (await axios.get(imgsrc)).data; if (imgsrc.startsWith("http")) {
image = Buffer.from(image).toString('base64'); image = (await axios.get(imgsrc)).data;
} image = Buffer.from(image).toString('base64');
else { }
if (imgsrc.startsWith("./")) { else {
imgsrc = imgsrc.replace("./", ""); if (imgsrc.startsWith("./")) {
} imgsrc = imgsrc.replace("./", "");
image = fs.readFileSync(`${this.templateDir}/${imgsrc}`, { encoding: "base64" }); }
} image = fs.readFileSync(`${this.templateDir}/${imgsrc}`, { encoding: "base64" });
}
image = `data:${img_type};base64,${image}`
$(element).attr("src", image) image = `data:${img_type};base64,${image}`
}) $(element).attr("src", image)
})
return $.html();
} return $.html();
}
/**
* This method manages the creation of pdfs via puppeteer. /**
* @param html The HTML that should get rendered. * This method manages the creation of pdfs via puppeteer.
* @param options Puppeteer PDF option (eg: {format: "A4"}) * @param html The HTML that should get rendered.
*/ * @param options Puppeteer PDF option (eg: {format: "A4"})
public async renderPdf(html: string, options): Promise<any> { */
html = await this.imgToBase64(html); public async renderPdf(html: string, options): Promise<any> {
let page = await this.browser.newPage(); html = await this.imgToBase64(html);
await page.setContent(html); let page = await this.browser.newPage();
const pdf = await page.pdf(options); await page.setContent(html);
await page.close(); const pdf = await page.pdf(options);
return pdf; await page.close();
} return pdf;
}
/**
* Merges multiple pdfs into one. /**
* @param pdfs The pdfs you want to merge as an buffer array. * Merges multiple pdfs into one.
* @returns The merged pdf as a buffer. * @param pdfs The pdfs you want to merge as an buffer array.
*/ * @returns The merged pdf as a buffer.
private async mergePdfs(pdfs: Buffer[]): Promise<Buffer> { */
const mergedPdf = await PDFDocument.create(); private async mergePdfs(pdfs: Buffer[]): Promise<Buffer> {
const mergedPdf = await PDFDocument.create();
for (const pdfBuffer of pdfs) {
const pdf = await PDFDocument.load(pdfBuffer); for (const pdfBuffer of pdfs) {
const copiedPages = await mergedPdf.copyPages(pdf, pdf.getPageIndices()); const pdf = await PDFDocument.load(pdfBuffer);
copiedPages.forEach((page) => { const copiedPages = await mergedPdf.copyPages(pdf, pdf.getPageIndices());
mergedPdf.addPage(page); copiedPages.forEach((page) => {
}); mergedPdf.addPage(page);
} });
}
return <Buffer>(await mergedPdf.save());
} return <Buffer>(await mergedPdf.save());
}
/**
* Generates a new dummy runner with halfspaces for all strings. /**
* Can be used to generate empty sponsoring contracts. * Generates a new dummy runner with halfspaces for all strings.
* @returns A new runner object that apears to be empty. * Can be used to generate empty sponsoring contracts.
*/ * @returns A new runner object that apears to be empty.
private generateEmptyRunner(): Runner { */
let group = new RunnerGroup(); private generateEmptyRunner(): Runner {
group.id = 0; let group = new RunnerGroup();
group.name = ""; group.id = 0;
let runner = new Runner(); group.name = "";
runner.id = 0; let runner = new Runner();
runner.firstname = ""; runner.id = 0;
runner.lastname = ""; runner.firstname = "";
runner.group = group; runner.lastname = "";
return runner; runner.group = group;
} return runner;
}
} }

View File

@ -1,16 +1,19 @@
import { config as configDotenv } from 'dotenv'; import { config as configDotenv } from 'dotenv';
configDotenv(); configDotenv();
export const config = { export const config = {
internal_port: parseInt(process.env.APP_PORT) || 4010, internal_port: parseInt(process.env.APP_PORT) || 4010,
development: process.env.NODE_ENV === "production", development: process.env.NODE_ENV === "production",
version: process.env.VERSION || require('../package.json').version version: process.env.VERSION || require('../package.json').version,
} eventname: process.env.EVENT_NAME || "Please set the event name",
let errors = 0 currency_symbol: process.env.CURRENCY_SYMBOL || "€",
if (typeof config.internal_port !== "number") { sponsoring_receipt_minimum_amount: process.env.SPONSORING_RECEIPT_MINIMUM_AMOUNT || "10"
errors++ }
} let errors = 0
if (typeof config.development !== "boolean") { if (typeof config.internal_port !== "number") {
errors++ errors++
} }
if (typeof config.development !== "boolean") {
errors++
}
export let e = errors export let e = errors