foliant
what: foliant使用
when: 2021/10/29
who：JRT
where：centos7
why:记录过程
说明：
foliant是一个一体化的文档创作工具。它允许您从单个标记源生成pdf和docx格式的独立文档以及网站。
foliant是一个higher order工具，这意味着它使用其他程序来完成它的工作。对于pdf和docx，它使用Pandoc，对于网站，它使用MkDocs。
叶型预处理器允许您在其他文档中包含部分文档，使用标志显示和隐藏内容，从文本呈现图表，等等。
使用：
1.拉取phthon镜像
docker pull python:3.8
2.进入容器
#使用python环境启动项目，端口默认8000，启动命令python -m http.server -d project，可通过指定port修改端口
#python -m http.server port(端口) -d project（项目名）
docker run -itd --name python -p 9001:8000 79372a158581
3.安装基础依赖
apt update -y
pip install foliant foliantcontrib.init
pip install foliantcontrib.mkdocs
pip install foliantcontrib.elasticsearch
#安装文件转换依赖
apt install -y texlive-full librsvg2-bin pandoc
pip install foliantcontrib.pandoc
4.创建项目(交互式输入项目名，如：jrt)
foliant init
5.用命令构建站点foliant make
foliant make site
6.您的网站在文件夹中生成，启动网络服务器查看它My_Project-2020-05-25.mkdocs
python -m http.server -d My_Project-2020-05-25.mkdocs
2.md转换为网页
1.将准备的md放入项目中src目录下
2.在foliant.yml中加入放入src目录的md名，例子如下：
title: Hello Foliant
chapters:
- index.md
+ - hello.md
3.构建站点：
foliant make site
4.启动服务：
python -m http.server -d My_Project-2020-05-25.mkdocs
3.md转换为pdf
在项目中执行foliant make pdf
结果如下：
root@fef857f6bb16:/opt/jrt# foliant make pdf
Parsing config... Done
Applying preprocessor flatten... Done
Applying preprocessor _unescape... Done
Making pdf with Pandoc... Done
────────────────────
Result:
jrt-2021-10-26.pdf
4.md转化为doc
root@fef857f6bb16:/opt/jrt# foliant make docx
Parsing config... Done
Applying preprocessor flatten... Done
Applying preprocessor _unescape... Done
Making docx with Pandoc... Done
────────────────────
Result:
jrt-2021-10-26.docx
root@fef857f6bb16:/opt/jrt# ls
Dockerfile docker-compose.yml jrt-2021-10-26.docx jrt-2021-10-26.pdf src
README.md foliant.yml jrt-2021-10-26.mkdocs requirements.txt
root@fef857f6bb16:/opt/jrt#
5.foliant搭配es实现全文检索配置
elasticsearch容器启动
#拉取容器
docker pull elasticsearch:7.9.3
#启动容器
docker run -itd --name jrt -p 8888：9200 -e "discovery.type=single-node" elasticsearch:7.9.3
elasticsearch服务端配置
1.解决跨域
#进入elasticsearch.yaml(增加以下内容)
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-credentials: true
#增加完后需要重启es生效
2.解决远程访问连接
#进入elasticsearch.yaml(增加以下内容)
network.host: 0.0.0.0
#增加完后需要重启es生效
3.创建索引，(测试索引名：jrt5，参考地址：http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html)
{"mappings": {
"properties": {
"url": {
"type": "text"
},
"title": {
"type": "text"
},
"content": {
"type": "text"
}
}
}
}
elasticsearch的网页端
1.搭建httpd或tomcat
2.下载项目
项目地址：https://github.com/foliant-docs/foliantcontrib.elasticsearch/blob/master/webapp_example
或
git clone https://github.com/foliant-docs/foliantcontrib.elasticsearch.git
3.修改项目
4.修改foliant
5.启动项目
修改项目
#进入foliant_elasticsearch.js
修改 const searchUrl = 'http://159.138.148.249:8889/jrt4/_search';
const baseUrl = 'http://159.138.148.249:9000/';
将 "query": textToSearch, 改为 "query": "*" + textToSearch,
#全文如下：
function performSearch(textToSearch) {
// Specify your Elasticsearch instance API URL here
const searchUrl = 'http://159.138.148.249:8889/jrt4/_search';
// Specify your site URL without trailing slash here
const baseUrl = 'http://159.138.148.249:9000/';
// Edit this query if needed. In this simple script, single API request is used for searching, and 50 first search results are shown. You may use AJAX to load more results dynamically
let query = {
"query": {
"multi_match": {
"query": "*" + textToSearch,
"type": "phrase_prefix",
"fields": [ "title^3", "content" ]
}
},
"highlight": {
"fields": {
"content": {}
}
},
"size": 50
};
let searchRequest = new XMLHttpRequest();
searchRequest.open('POST', searchUrl, true);
searchRequest.setRequestHeader('Content-Type', 'application/json; charset=utf-8');
searchRequest.onload = function() {
let response = JSON.parse(searchRequest.responseText);
document.getElementById('foliant_elasticsearch_total').innerHTML = '<p class="foliant_elasticsearch_success">Results: ' + response.hits.total.value + '</p>';
let output = '';
for(let i = 0; i < response.hits.hits.length; i++) {
output += '<h2>' + response.hits.hits[i]._source.title + '</h2><p>Page URL: <a href="' + baseUrl + response.hits.hits[i]._source.url + '">' + baseUrl + response.hits.hits[i]._source.url + '</a></p><pre>';
for(let j = 0; j < response.hits.hits[i].highlight.content.length; j++) {
output += response.hits.hits[i].highlight.content[j] + '\n\n';
}
output += '</pre>';
}
document.getElementById('foliant_elasticsearch_results').innerHTML = output;
};
searchRequest.onerror = function() {
document.getElementById('foliant_elasticsearch_total').innerHTML = '<p class="foliant_elasticsearch_error">Error</p>';
};
searchRequest.send(JSON.stringify(query));
}
folient配置
#修改foliant.yml
index_properties: {}增加：
"settings": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "ngram",
"min_gram": 1,
"max_gram": 10
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter"
]
}
}
},
"index.max_ngram_diff" : 10
},
"mappings": {
"properties": {
"title": {
"type": "text",
"analyzer": "autocomplete",
"search_analyzer": "standard"
},
"content": {
"type": "text",
"analyzer": "autocomplete",
"search_analyzer": "standard"
}
}
}
#全文如下：
title: jrt
chapters:
- index.md
- note.md
- a.md
- note1.md
preprocessors:
- elasticsearch:
es_url: 'http://159.138.148.249:8889'
index_name: 'jrt5'
index_copy_name: ''
index_properties: {
"settings": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "ngram",
"min_gram": 1,
"max_gram": 10
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter"
]
}
}
},
"index.max_ngram_diff" : 10
},
"mappings": {
"properties": {
"title": {
"type": "text",
"analyzer": "autocomplete",
"search_analyzer": "standard"
},
"content": {
"type": "text",
"analyzer": "autocomplete",
"search_analyzer": "standard"
}
}
}
}
actions:
- delete
- create
use_chapters: true
format: plaintext
escape_html: true
url_transform:
- '\/?index\.md$': '/'
- '\.md$': '/'
- '^([^\/]+)': '/\g<1>'
require_env: false
targets: []
6.延申
IK分词器
源码：（https://github.com/medcl/elasticsearch-analysis-ik/ ）
git clone https://github.com/medcl/elasticsearch-analysis-ik
cd elasticsearch-analysis-ik
git checkout tags/{version}
mvn clean
mvn compile
mvn package
#编译后的包在/target/releases/elasticsearch-analysis-ik-*.zip，解压 到你的 elasticsearch 的插件目录, 如: plugins/ik下，若plugins下无ik，创建该文件夹， 重启elasticsearch
#ik版本必须与es版本保持一致，若不一致，解决方法是：https://blog.csdn.net/love__guo/article/details/111197562
es常用命令
查索引有哪些 ip：端口/_cat/indices?v
查类型 ip：端口/索引名/_mapping?pretty=true
查结构 ip：端口/索引名/_mapping?pretty
删除索引 delete ip：端口/索引名
优化后结果
title: yxl
slug: yxl
chapters:
- index.md
preprocessors:
- elasticsearch:
es_url: 'http://159.138.29.132:9002'
index_name: 'yxl'
index_copy_name: ''
index_properties: {
"settings": {
"analysis": {
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "my_tokenizer",
"filter": [
"lowercase"
]
}
},
"tokenizer": {
"my_tokenizer": {
"type": "ngram",
"min_gram": 3,
"max_gram": 3,
"token_chars": [
"letter",
"digit",
"symbol",
"punctuation"
]
}
}
}
}
}
actions:
- delete
- create
use_chapters: false
format: plaintext
escape_html: true
url_transform:
- '\/?index\.md$': '/'
- '\.md$': '/'
- '^([^\/]+)': '/\g<1>'
require_env: false
targets: []
Dockerfile
FROM python:3.8
LABEL JRT test
RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list && \
apt update -y && \
pip config set global.index-url http://mirrors.aliyun.com/pypi/simple/ && \
pip config set install.trusted-host mirrors.aliyun.com && \
pip config set global.timeout 6000 && \
pip install foliant foliantcontrib.init && \
pip install foliantcontrib.mkdocs && \
pip install foliantcontrib.elasticsearch
WORKDIR /
EXPOSE 8000
CMD ["&"]