lfk
/
document-server
2
0
Fork 0
Code Issues Pull Requests Projects Releases 13 Wiki Activity

Merge pull request 'Alpha Release 0.1.3 - More env vars' (#20) from dev into main
continuous-integration/drone/push Build is failing Details
continuous-integration/drone/tag Build is failing Details 

Reviewed-on: #20
Reviewed-by: Philipp Dormann <philipp@philippdormann.de>
This commit is contained in:
Nicolai Ort 2021-02-09 16:43:06 +00:00
parent 123cf8ad48 ad9a8a4fe0
commit ecd02a1af7
5 changed files with 308 additions and 287 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
CHANGELOG.md
View File

@ -2,8 +2,22 @@
All notable changes to this project will be documented in this file. Dates are displayed in UTC.
#### [v0.1.3](https://git.odit.services/lfk/document-server/compare/v0.1.2...v0.1.3)
- 🚀Bumped version to v0.1.3 [`6a14232`](https://git.odit.services/lfk/document-server/commit/6a142328898d5b89fa11eaf033372971d1093b0c)
- 🧾New changelog file version [CI SKIP] [skip ci] [`b6296b8`](https://git.odit.services/lfk/document-server/commit/b6296b8d97cda943dfb5e11bc9dfbb2f363f5b81)
- Merge pull request 'Load more stuff from env feature/16-env_vars' (#17) from feature/16-env_vars into dev [`bc4d16e`](https://git.odit.services/lfk/document-server/commit/bc4d16e6f8959ed35d7e87647de84584cdfddd7b)
- Added new env vars to config [`3bb322e`](https://git.odit.services/lfk/document-server/commit/3bb322ede5db15a147c0d7a8db2a68ccb7fa2112)
- Added new env vars to readme [`b77bb3a`](https://git.odit.services/lfk/document-server/commit/b77bb3ad9dba9d73c2c81215ba57936192155a9a)
- Now loading interpolation vars from config/env [`b4ebae2`](https://git.odit.services/lfk/document-server/commit/b4ebae283b472b2f0c6e28caed49b30edb119585)
- 🧾New changelog file version [CI SKIP] [skip ci] [`a306009`](https://git.odit.services/lfk/document-server/commit/a30600943d01116b99e946cb705a16d0372b5095)
#### [v0.1.2](https://git.odit.services/lfk/document-server/compare/v0.1.1...v0.1.2)
> 7 February 2021
- Merge pull request 'Alpha Release 0.1.2 - Hotfix release' (#15) from dev into main [`123cf8a`](https://git.odit.services/lfk/document-server/commit/123cf8ad48a45fa10dcd5208215a6e525f31115a)
- 🧾New changelog file version [CI SKIP] [skip ci] [`22b1e00`](https://git.odit.services/lfk/document-server/commit/22b1e0097efc865de9cc150cb0d0b99bf789b519)
- 🚀Bumped version to v0.1.2 [`7e507d4`](https://git.odit.services/lfk/document-server/commit/7e507d4cc415877ac0b25503dc0ff9ecdceabf42)
- PAtch: Copy locales [`f7dfd6d`](https://git.odit.services/lfk/document-server/commit/f7dfd6d0c3c69881338bc1f66d5d33ae9abff628)

151
README.md
View File

@ -1,75 +1,78 @@
# @lfk/document-server
The document generation server responsible for creating pdfs for sponsoring contracts, certificates and more.
This server doesn't interact with any database and can therefor be deployed on it's own.
The basic generation mechanism makes the templates and routes interchangeable (if you want to expand or modify it).
## Quickstart 🐳
> Use this to run the document server in docker.
1. Clone the repo or copy the docker-compose
2. Run in the folder that contains the docker-compose file: `docker-compose up -d`
3. Visit http://127.0.0.1:4010/docs to check if the server is running
## Dev Setup 🛠
> Local dev setup
1. Rename the .env.example file to .env (you can adjust app port and other settings, if needed)
2. Install Dependencies
```bash
yarn
```
3. Start the server
```bash
yarn dev
```
## ENV Vars
> You can provide them via .env file or docker env vars.
| Name | Type | Default | Description
| - | - | - | -
| APP_PORT | Number | 4010 | The port the backend server listens on. Is optional.
| NODE_ENV | String | dev | The apps env - influences debug info.
## Templates
> The document server uses html templates to generate various pdf documents.
> The templates are stored in src/templates by default.
We provide a set of default templates that we use for the ["Lauf für Kaya!" charity run](https://lauf-fuer-kaya.de).
We use handlebars for templateing utilizing i18next for translation - the i18n string format in the templates is : `{{__ "string"}}`
You can provide your own templates by replacing the ones we provided before compiling the project or by simply mounting your custom templates into the docker container.
The server currently needs the following templates to work:
* sponsoring_contract.html
### Sponsoring Contracts
| Template Data | Type | Optional | Description
| - | - | - | -
| runners | array(Runner) | ❌ | The runner objects. We generate a contract for each runner on a new DIN-A5 page.
## Recommended Editor
[Visual Studio Code](https://code.visualstudio.com/)
### Recommended Extensions
* will be automatically recommended via ./vscode/extensions.json
* we also provide a config for i18n-ally in the .vscode folder
## Staging
### Branches & Tags
* vX.Y.Z: Release tags created from the main branch
* The version numbers follow the semver standard
* A new release tag automaticly triggers the release ci pipeline
* main: Protected "release" branch
* The latest tag of the docker image get's build from this
* New releases get created as tags from this
* dev: Current dev branch for merging the different feature branches and bugfixes
* The dev tag of the docker image get's build from this
* Only push minor changes to this branch!
* To merge a feature branch into this please create a pull request
* feature/xyz: Feature branches - nameing scheme: `feature/issueid-title`
# @lfk/document-server
The document generation server responsible for creating pdfs for sponsoring contracts, certificates and more.
This server doesn't interact with any database and can therefor be deployed on it's own.
The basic generation mechanism makes the templates and routes interchangeable (if you want to expand or modify it).
## Quickstart 🐳
> Use this to run the document server in docker.
1. Clone the repo or copy the docker-compose
2. Run in the folder that contains the docker-compose file: `docker-compose up -d`
3. Visit http://127.0.0.1:4010/docs to check if the server is running
## Dev Setup 🛠
> Local dev setup
1. Rename the .env.example file to .env (you can adjust app port and other settings, if needed)
2. Install Dependencies
```bash
yarn
```
3. Start the server
```bash
yarn dev
```
## ENV Vars
> You can provide them via .env file or docker env vars.
| Name | Type | Default | Description
| - | - | - | -
| APP_PORT | Number | 4010 | The port the backend server listens on. Is optional.
| NODE_ENV | String | dev | The apps env - influences debug info.
| EVENT_NAME | String | "Please set the event name" | The event's name - used to generate pdf text.
| CURRENCY_SYMBOL | String | "€" | The your currency's symbol - used to generate pdf text.
| SPONSORING_RECEIPT_MINIMUM_AMOUNT | String | "10" | The mimimum total donation amount a sponsor has to donate to be able to receive a donation receipt - used to generate pdf text.
## Templates
> The document server uses html templates to generate various pdf documents.
> The templates are stored in src/templates by default.
We provide a set of default templates that we use for the ["Lauf für Kaya!" charity run](https://lauf-fuer-kaya.de).
We use handlebars for templateing utilizing i18next for translation - the i18n string format in the templates is : `{{__ "string"}}`
You can provide your own templates by replacing the ones we provided before compiling the project or by simply mounting your custom templates into the docker container.
The server currently needs the following templates to work:
* sponsoring_contract.html
### Sponsoring Contracts
| Template Data | Type | Optional | Description
| - | - | - | -
| runners | array(Runner) | ❌ | The runner objects. We generate a contract for each runner on a new DIN-A5 page.
## Recommended Editor
[Visual Studio Code](https://code.visualstudio.com/)
### Recommended Extensions
* will be automatically recommended via ./vscode/extensions.json
* we also provide a config for i18n-ally in the .vscode folder
## Staging
### Branches & Tags
* vX.Y.Z: Release tags created from the main branch
* The version numbers follow the semver standard
* A new release tag automaticly triggers the release ci pipeline
* main: Protected "release" branch
* The latest tag of the docker image get's build from this
* New releases get created as tags from this
* dev: Current dev branch for merging the different feature branches and bugfixes
* The dev tag of the docker image get's build from this
* Only push minor changes to this branch!
* To merge a feature branch into this please create a pull request
* feature/xyz: Feature branches - nameing scheme: `feature/issueid-title`
* bugfix/xyz: Branches for bugfixes - nameing scheme:`bugfix/issueid-title`

2
package.json
View File

@ -1,6 +1,6 @@
{
"name": "@odit/lfk-document-server",
"version": "0.1.2",
"version": "0.1.3",
"description": "The document generation server for the LfK! runner system. This generates certificates, sponsoring aggreements and more",
"main": "src/app.ts",
"scripts": {

395
src/PdfCreator.ts
View File

@ -1,198 +1,199 @@
import axios from 'axios';
import cheerio from "cheerio";
import fs from "fs";
import Handlebars from 'handlebars';
import i18next from "i18next";
import Backend from 'i18next-fs-backend';
import mime from "mime-types";
import path from 'path';
import { PDFDocument } from 'pdf-lib';
import puppeteer from "puppeteer";
import { Runner } from './models/Runner';
import { RunnerGroup } from './models/RunnerGroup';
/**
* This class is responsible for all things pdf creation.
* This uses the html templates from src/templates.
*/
export class PdfCreator {
private templateDir = path.join(__dirname, '/templates');
private browser;
private static interpolations = { eventname: "Lauf für Kaya! 2021", sponsoring_receipt_minimum_amount: '10', currency_symbol: "€" }
/**
* Main constructor.
* Initializes i18n(ext), Handlebars and puppeteer.
*/
constructor() {
this.init();
}
/**
* Main constructor.
* Initializes i18n(ext), Handlebars and puppeteer.
*/
public async init() {
const minimal_args = [
'--autoplay-policy=user-gesture-required',
'--disable-background-networking',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-update',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-domain-reliability',
'--disable-extensions',
'--disable-features=AudioServiceOutOfProcess',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-notifications',
'--disable-offer-store-unmasked-wallet-cards',
'--disable-popup-blocking',
'--disable-print-preview',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-speech-api',
'--disable-sync',
'--hide-scrollbars',
'--ignore-gpu-blacklist',
'--metrics-recording-only',
'--mute-audio',
'--no-default-browser-check',
'--no-first-run',
'--no-pings',
'--no-zygote',
'--password-store=basic',
'--use-gl=swiftshader',
'--no-sandbox'
];
await i18next
.use(Backend)
.init({
fallbackLng: 'en',
lng: 'en',
backend: {
loadPath: path.join(__dirname, '/locales/{{lng}}.json')
}
});
await Handlebars.registerHelper('__',
function (str) {
return i18next.t(str, PdfCreator.interpolations).toString();
}
);
this.browser = await puppeteer.launch({ headless: true, args: minimal_args });
}
/**
* Generate sponsoring contract pdfs.
* @param runner The runner you want to generate the contracts for.
* @param locale The locale used for the contracts (default:en)
*/
public async generateSponsoringContract(runners: Runner[], locale: string = "en"): Promise<Buffer> {
if (runners.length == 1 && Object.keys(runners[0]).length == 0) {
runners[0] = this.generateEmptyRunner();
}
if (runners.length > 50) {
let pdf_promises = new Array<Promise<Buffer>>();
let i, j;
for (i = 0, j = runners.length; i < j; i += 50) {
let chunk = runners.slice(i, i + 50);
pdf_promises.push(this.generateSponsoringContract(chunk, locale));
}
const pdfs = await Promise.all(pdf_promises);
return await this.mergePdfs(pdfs);
}
await i18next.changeLanguage(locale);
const template_source = fs.readFileSync(`${this.templateDir}/sponsoring_contract.html`, 'utf8');
const template = Handlebars.compile(template_source);
const result = template({ runners })
const pdf = await this.renderPdf(result, { format: "A5", landscape: true });
return pdf
}
/**
* Converts all images in html to base64.
* Works with image files in the template directory or images from urls.
* @param html The html string whoms images shall get replaced.
*/
public async imgToBase64(html): Promise<string> {
const $ = cheerio.load(html)
$('img').each(async (index, element) => {
let imgsrc = $(element).attr("src");
const img_type = mime.lookup(imgsrc);
if (!(img_type.includes("image"))) {
throw new Error("File is not image mime type");
}
let image;
if (imgsrc.startsWith("http")) {
image = (await axios.get(imgsrc)).data;
image = Buffer.from(image).toString('base64');
}
else {
if (imgsrc.startsWith("./")) {
imgsrc = imgsrc.replace("./", "");
}
image = fs.readFileSync(`${this.templateDir}/${imgsrc}`, { encoding: "base64" });
}
image = `data:${img_type};base64,${image}`
$(element).attr("src", image)
})
return $.html();
}
/**
* This method manages the creation of pdfs via puppeteer.
* @param html The HTML that should get rendered.
* @param options Puppeteer PDF option (eg: {format: "A4"})
*/
public async renderPdf(html: string, options): Promise<any> {
html = await this.imgToBase64(html);
let page = await this.browser.newPage();
await page.setContent(html);
const pdf = await page.pdf(options);
await page.close();
return pdf;
}
/**
* Merges multiple pdfs into one.
* @param pdfs The pdfs you want to merge as an buffer array.
* @returns The merged pdf as a buffer.
*/
private async mergePdfs(pdfs: Buffer[]): Promise<Buffer> {
const mergedPdf = await PDFDocument.create();
for (const pdfBuffer of pdfs) {
const pdf = await PDFDocument.load(pdfBuffer);
const copiedPages = await mergedPdf.copyPages(pdf, pdf.getPageIndices());
copiedPages.forEach((page) => {
mergedPdf.addPage(page);
});
}
return <Buffer>(await mergedPdf.save());
}
/**
* Generates a new dummy runner with halfspaces for all strings.
* Can be used to generate empty sponsoring contracts.
* @returns A new runner object that apears to be empty.
*/
private generateEmptyRunner(): Runner {
let group = new RunnerGroup();
group.id = 0;
group.name = " ";
let runner = new Runner();
runner.id = 0;
runner.firstname = " ";
runner.lastname = " ";
runner.group = group;
return runner;
}
import axios from 'axios';
import cheerio from "cheerio";
import fs from "fs";
import Handlebars from 'handlebars';
import i18next from "i18next";
import Backend from 'i18next-fs-backend';
import mime from "mime-types";
import path from 'path';
import { PDFDocument } from 'pdf-lib';
import puppeteer from "puppeteer";
import { config } from './config';
import { Runner } from './models/Runner';
import { RunnerGroup } from './models/RunnerGroup';
/**
* This class is responsible for all things pdf creation.
* This uses the html templates from src/templates.
*/
export class PdfCreator {
private templateDir = path.join(__dirname, '/templates');
private browser;
private static interpolations = { eventname: config.eventname, sponsoring_receipt_minimum_amount: config.sponsoring_receipt_minimum_amount, currency_symbol: config.currency_symbol }
/**
* Main constructor.
* Initializes i18n(ext), Handlebars and puppeteer.
*/
constructor() {
this.init();
}
/**
* Main constructor.
* Initializes i18n(ext), Handlebars and puppeteer.
*/
public async init() {
const minimal_args = [
'--autoplay-policy=user-gesture-required',
'--disable-background-networking',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-breakpad',
'--disable-client-side-phishing-detection',
'--disable-component-update',
'--disable-default-apps',
'--disable-dev-shm-usage',
'--disable-domain-reliability',
'--disable-extensions',
'--disable-features=AudioServiceOutOfProcess',
'--disable-hang-monitor',
'--disable-ipc-flooding-protection',
'--disable-notifications',
'--disable-offer-store-unmasked-wallet-cards',
'--disable-popup-blocking',
'--disable-print-preview',
'--disable-prompt-on-repost',
'--disable-renderer-backgrounding',
'--disable-speech-api',
'--disable-sync',
'--hide-scrollbars',
'--ignore-gpu-blacklist',
'--metrics-recording-only',
'--mute-audio',
'--no-default-browser-check',
'--no-first-run',
'--no-pings',
'--no-zygote',
'--password-store=basic',
'--use-gl=swiftshader',
'--no-sandbox'
];
await i18next
.use(Backend)
.init({
fallbackLng: 'en',
lng: 'en',
backend: {
loadPath: path.join(__dirname, '/locales/{{lng}}.json')
}
});
await Handlebars.registerHelper('__',
function (str) {
return i18next.t(str, PdfCreator.interpolations).toString();
}
);
this.browser = await puppeteer.launch({ headless: true, args: minimal_args });
}
/**
* Generate sponsoring contract pdfs.
* @param runner The runner you want to generate the contracts for.
* @param locale The locale used for the contracts (default:en)
*/
public async generateSponsoringContract(runners: Runner[], locale: string = "en"): Promise<Buffer> {
if (runners.length == 1 && Object.keys(runners[0]).length == 0) {
runners[0] = this.generateEmptyRunner();
}
if (runners.length > 50) {
let pdf_promises = new Array<Promise<Buffer>>();
let i, j;
for (i = 0, j = runners.length; i < j; i += 50) {
let chunk = runners.slice(i, i + 50);
pdf_promises.push(this.generateSponsoringContract(chunk, locale));
}
const pdfs = await Promise.all(pdf_promises);
return await this.mergePdfs(pdfs);
}
await i18next.changeLanguage(locale);
const template_source = fs.readFileSync(`${this.templateDir}/sponsoring_contract.html`, 'utf8');
const template = Handlebars.compile(template_source);
const result = template({ runners })
const pdf = await this.renderPdf(result, { format: "A5", landscape: true });
return pdf
}
/**
* Converts all images in html to base64.
* Works with image files in the template directory or images from urls.
* @param html The html string whoms images shall get replaced.
*/
public async imgToBase64(html): Promise<string> {
const $ = cheerio.load(html)
$('img').each(async (index, element) => {
let imgsrc = $(element).attr("src");
const img_type = mime.lookup(imgsrc);
if (!(img_type.includes("image"))) {
throw new Error("File is not image mime type");
}
let image;
if (imgsrc.startsWith("http")) {
image = (await axios.get(imgsrc)).data;
image = Buffer.from(image).toString('base64');
}
else {
if (imgsrc.startsWith("./")) {
imgsrc = imgsrc.replace("./", "");
}
image = fs.readFileSync(`${this.templateDir}/${imgsrc}`, { encoding: "base64" });
}
image = `data:${img_type};base64,${image}`
$(element).attr("src", image)
})
return $.html();
}
/**
* This method manages the creation of pdfs via puppeteer.
* @param html The HTML that should get rendered.
* @param options Puppeteer PDF option (eg: {format: "A4"})
*/
public async renderPdf(html: string, options): Promise<any> {
html = await this.imgToBase64(html);
let page = await this.browser.newPage();
await page.setContent(html);
const pdf = await page.pdf(options);
await page.close();
return pdf;
}
/**
* Merges multiple pdfs into one.
* @param pdfs The pdfs you want to merge as an buffer array.
* @returns The merged pdf as a buffer.
*/
private async mergePdfs(pdfs: Buffer[]): Promise<Buffer> {
const mergedPdf = await PDFDocument.create();
for (const pdfBuffer of pdfs) {
const pdf = await PDFDocument.load(pdfBuffer);
const copiedPages = await mergedPdf.copyPages(pdf, pdf.getPageIndices());
copiedPages.forEach((page) => {
mergedPdf.addPage(page);
});
}
return <Buffer>(await mergedPdf.save());
}
/**
* Generates a new dummy runner with halfspaces for all strings.
* Can be used to generate empty sponsoring contracts.
* @returns A new runner object that apears to be empty.
*/
private generateEmptyRunner(): Runner {
let group = new RunnerGroup();
group.id = 0;
group.name = " ";
let runner = new Runner();
runner.id = 0;
runner.firstname = " ";
runner.lastname = " ";
runner.group = group;
return runner;
}
}

33
src/config.ts
View File

@ -1,16 +1,19 @@
import { config as configDotenv } from 'dotenv';
configDotenv();
export const config = {
internal_port: parseInt(process.env.APP_PORT) || 4010,
development: process.env.NODE_ENV === "production",
version: process.env.VERSION || require('../package.json').version
}
let errors = 0
if (typeof config.internal_port !== "number") {
errors++
}
if (typeof config.development !== "boolean") {
errors++
}
import { config as configDotenv } from 'dotenv';
configDotenv();
export const config = {
internal_port: parseInt(process.env.APP_PORT) || 4010,
development: process.env.NODE_ENV === "production",
version: process.env.VERSION || require('../package.json').version,
eventname: process.env.EVENT_NAME || "Please set the event name",
currency_symbol: process.env.CURRENCY_SYMBOL || "€",
sponsoring_receipt_minimum_amount: process.env.SPONSORING_RECEIPT_MINIMUM_AMOUNT || "10"
}
let errors = 0
if (typeof config.internal_port !== "number") {
errors++
}
if (typeof config.development !== "boolean") {
errors++
}
export let e = errors