175 lines
28 KiB
Plaintext
175 lines
28 KiB
Plaintext
|
+
|
|||
|
|
ô[!jVV ã óR a€ 0 t $ ^ RIHt ^ RIt^ RIt^ RIt^ RIt^ RIt^ RIt^ RI t ^ RI
|
|||
|
|
H t H t H
|
|||
|
|
t
|
|||
|
|
^ RIt^ RIHtHtHtHt ^ RIt^ RIHt ^ RIHt ^ RIHt ^ RI H t ^ R I H t ^ R
|
|||
|
|
I H!t" ]PF PI R R 4 t%]PF PI R
|
|||
|
|
R4 t&]PF PI RR4 PO 4 R<9 t(]PF PI RR4 t)]PF PI RR4 t*Rs+R],R& ]-! 4 t.R= F t/].Pa ] ! ]/4 4 K ]1! ]. U u0 uF q Pe R4 kK up 4 t3]PF PI RR4 t4]PF PI RR4 PO 4 R<9 t5]PF PI R R4 t6]PF PI R R4 t7]PF PI R R4 t8R t9R R! lt:R" R# lt;R$ R% lt<R& R' lt=]=! 4 t>]?R(8X Ed ]9! 4 ]! 4 s+]@! ]PF PI R)R*4 4 tA]PF PI R+R,4 tB]C! R>4 ]C! R-4 ]C! R.]% 24 ]C! R/]& 24 ]C! R0]( 24 ]C! R1]);' g R2 24 ]C! R3]* 24 ]C! R4]B R5]A 24 ]C! R6R7P‰ ]34 24 ]C! R>4 ]>P‹ ]B]A]PF PI R8R94 R8H R:R;7 R# R# u up i )?é )Ú annotationsN)ÚAnyÚDictÚOptional)ÚFlaskÚjsonifyÚrequestÚResponse)Ú ConfigParser)Ú PdfConverter)Úcreate_model_dict)Útext_from_rendered)Úload_extensions)ÚsettingsÚ OLLAMA_HOSTzhttp://10.0.1.127:11434ÚDEESEEK_OCR_MODELz deepseek-ocrÚ AMD_COMPUTEÚfalseÚ1Ú TORCH_DEVICEÚ Ú MODEL_DTYPEÚfloat32zOptional[Dict[str, Any]]Ú _marker_dictÚ.Ú LLM_SERVICEz$marker.services.ollama.OllamaServiceÚUSE_LLMÚOPENAI_BASE_URLÚOPENAI_API_KEYÚ OPENAI_MODELc ó‚ € \ ' dF \ ' g: R\ P R& R\ P R& R\ P R& \ R8X d R\ P R& \ P P R4 ;p ' d) V \ P R& V \ P R& R# R# ) z9Apply AMD GPU / environment overrides before torch loads.Úcudar ÚTORCH_DEVICE_MODELz9.0.6ÚHSA_OVERRIDE_GFX_VERSIONÚbfloat16r N)r r ÚosÚenvironr Úget)Útorch_device_overrides Ú%/home/oval/Projects/marker-api/app.pyÚ_configure_envr+ 0 sˆ € çƒ{Ÿ<›<Ø%+Œ�
|
|||
|
|
‰
|
|||
|
|
�>Ñ"Ø+1Œ�
|
|||
|
|
‰
|
|||
|
|
Ð'Ñ(Ø18Œ�
|
|||
|
|
‰
|
|||
|
|
Ð-Ñ.Ü�jÔ Ø$.Œ�
|
|||
|
|
‰
|
|||
|
|
�=Ñ!Ü "§
|
|||
|
|
¡
|
|||
|
|
§¡¨~Ó >Ð>Ð Ö>Ø%:Œ�
|
|||
|
|
‰
|
|||
|
|
�>Ñ"Ø+@Œ�
|
|||
|
|
‰
|
|||
|
|
Ð'Ó(ñ ?ó c ó € V ^8„ d QhRR/# )é ÚreturnúDict[str, Any]© )Úformats "r* Ú __annotate__r3 = s € ÷ ñ ˜ñ r, c ó2 € \ f \ 4 s \ # )N)r r
|
|||
|
|
r1 r, r* Úget_model_dictr5 = s € äÒÜ(Ó*ˆ Ü Ðr, c ó € V ^8„ d QhRRRR/# )r. Úextrar r/ r0 r1 )r2 s "r* r3 r3 D s € ÷ 0ñ 0˜3ð 0 >ñ 0r, c óò € / pV P 4 F w r#Vf K W1V& K VP RR4 VP RR4 VP RR4 VP RR 4 VP RR4 VP R R4 VP R
|
|||
|
|
\ P 4 VP R \ 4 VP R \
|
|||
|
|
4 \
|
|||
|
|
V4 pVP 4 # )
|
|||
|
|
NÚ
|
|||
|
|
output_formatÚmarkdownÚ force_ocrFÚpaginate_outputÚ
|
|||
|
|
page_rangeÚdisable_multiprocessingTÚdisable_image_extractionÚ
|
|||
|
|
output_dirÚ llm_serviceÚuse_llm)ÚitemsÚ
|
|||
|
|
setdefaultÚmarker_settingsÚ
|
|||
|
|
OUTPUT_DIRÚDEFAULT_LLM_SERVICEÚDEFAULT_USE_LLMr Úgenerate_config_dict)r7 ÚoptsÚkÚvÚ
|
|||
|
|
config_parsers , r* Ú
|
|||
|
|
build_optionsrN D sÈ € Ø €DØ— ‘ –
|
|||
|
|
‰ˆØ Œ=Ø�‹Gñ ð
|
|||
|
|
‡O�O�O ZÔ0؇O�O�K Ô'؇O�OÐ% uÔ-؇O�O�L $Ô'؇O�OÐ-¨tÔ4؇O�OÐ.°Ô6؇O�O�L¤/×"<Ñ"<Ô=؇O�O�MÔ#6Ô7؇O�O�IœÔ/ä Ó&€MØ × -Ñ -Ó /Ð/r, c ó( € V ^8„ d QhRRRRRRRR/# ) r. Ú raw_bytesÚbytesÚfilenameÚstrrJ r r/ Údictr1 )r2 s "r* r3 r3 Y s( € ÷ .ñ . %ð .°3ð .Àð .Èñ .r, c
|
|||
|
|
ól € \ P ! RRV9 d! RVP R^4 R, , MRR7 p VP V 4 VP 4 VP
|
|||
|
|
p\
|
|||
|
|
R/ VB pRVR& \ 4 p\ VP 4 UUu/ uF
|
|||
|
|
w rxVf K WxbK upp4 p \ VVV P 4 V P 4 V P 4 R7 p
|
|||
|
|
V
|
|||
|
|
! V4 p \ V 4 w rÍp TP 4 \& P( P+ TP
|
|||
|
|
4 ' d! \& P, ! TP
|
|||
|
|
4 / p\. P0 pTP 4 Fd w pp\2 P4 ! 4 pTP7 TTR 7 \8 P: ! TP= 4 4 P? R 4 TT& Kf R
|
|||
|
|
TPA RR4 RT RTRR\C 4 9 d T PD R R/# / R R/# u uppi \ d� p\ P" ! 4 R RR
|
|||
|
|
\% T4 /u Rp?TP 4 \& P( P+ TP
|
|||
|
|
4 ' d" \& P, ! TP
|
|||
|
|
4 # # Rp?ii ; i TP 4 \& P( P+ TP
|
|||
|
|
4 ' d" \& P, ! TP
|
|||
|
|
4 i i ; i)z;Convert in-memory file bytes to markdown (or other format).Fr r )ÚdeleteÚsuffixTÚ disable_tqdmN)ÚconfigÚ
|
|||
|
|
artifact_dictÚprocessor_listÚrendererrA ÚsuccessÚerror)r2 zutf-8r2 r9 r: ÚoutputÚ
|
|||
|
|
images_b64ÚmetadataÚrenderedéÿÿÿÿr1 )#ÚtempfileÚNamedTemporaryFileÚrsplitÚwriteÚcloseÚnamerN r5 r rC r Úget_processorsÚ get_rendererÚget_llm_servicer Ú ExceptionÚ tracebackÚ print_excrS r& ÚpathÚexistsÚremoverE ÚOUTPUT_IMAGE_FORMATÚioÚBytesIOÚsaveÚbase64Ú b64encodeÚgetvalueÚdecoder( Údirra )rP rR rJ ÚtmpÚfilepathÚ config_dictÚ
|
|||
|
|
model_dictrK rL ÚparsedÚ converterrb ÚtextÚ_ÚimagesÚexcÚencoded_imagesÚimg_fmtÚimgÚbufs &&, r* Úconvert_file_bytesrŠ Y s� € ô ×
|
|||
|
|
%Ò
|
|||
|
|
%¨UÐ`cÐgoÔ`o¸3ÀÇ Á ÐQTÐVWÓAXÐY[ÕA\Ö;\ÐuwÔ
|
|||
|
|
x€Cð Ø � ‰ �)Ô Ø � ‰ Œ Ø—8‘8ˆä#Ñ+ dÑ+ˆ Ø&*ˆ �NÑ#ä#Ó%ˆ
|
|||
|
|
Ü °·
|
|||
|
|
±
|
|||
|
|
´ Ô N± ©¨Àœt˜qšt± Ò NÓOˆä Ø Ø$Ø!×0Ñ0Ó2Ø×(Ñ(Ó*Ø ×.Ñ.Ó0ô
|
|||
|
|
ˆ ñ ˜XÓ&ˆÜ,¨XÓ6‰ˆ‘ð � ‰ Œ Ü
|
|||
|
|
�7‰7�>‰>˜#Ÿ(™(× #Ò #Ü �IŠI�c—h‘hÔ à%'€NÜ ×1Ñ1€GØ—,‘,–.‰ˆˆ3Ü�jŠj‹lˆØ �‰�˜WˆÔ%Ü "× ,Ò ,¨S¯\©\«^Ó <× CÑ CÀGÓ Lˆ�qÓñ !ð �$—(‘(˜?¨JÓ7Ø�$Ø�nب´s³uÔ)<�H×%Ñ%Ø�4ð ð ð CEØ�4ð ð ùó7 Oøô ô 5Ü×ÒÔ Ø˜5 '¬3¨s«8Ð4Ô4à � ‰ Œ Ü
|
|||
|
|
�7‰7�>‰>˜#Ÿ(™(× #Ò #Ü �IŠI�c—h‘hÕ ð $ûð 5ûð � ‰ Œ Ü
|
|||
|
|
�7‰7�>‰>˜#Ÿ(™(× #Ò #Ü �IŠI�c—h‘hÕ ð $úsK ÁA H5 Â H/
|
|||
|
|
Â-H/
|
|||
|
|
Â3AH5 È/H5 È5 KÉ #K É#KÉ$K Ë KËK ËA"L3c ó € V ^8„ d QhRR/# )r. r/ r r1 )r2 s "r* r3 r3 Š s € ÷ Cñ C�Eñ Cr, c ó( a€ \ \ 4 p R oV P R4 V3R l4 pV P R4 R 4 pV P RR.R7 R 4 pV P R R.R7 R
|
|||
|
|
4 pV P R R.R7 R 4 pV # )
|
|||
|
|
ad <html><head><title>marker-api</title>
|
|||
|
|
<style>body{font-family:sans-serif;max-width:900px;margin:40px auto;padding:0 20px}
|
|||
|
|
pre{background:#f4f4f4;padding:12px;border-radius:4px;overflow-x:auto}
|
|||
|
|
table{border-collapse:collapse;width:100%;margin:16px 0}th,td{border:1px solid #ccc;padding:8px;text-align:left}
|
|||
|
|
th{background:#f0f0f0}</style></head><body>
|
|||
|
|
<h1>marker-api</h1>
|
|||
|
|
<p>Convert PDFs, EPUBs, DOCX, XLSX, PPTX, HTML, and images to Markdown.</p>
|
|||
|
|
<h2>Endpoints</h2>
|
|||
|
|
<table>
|
|||
|
|
<tr><th>Endpoint</th><th>Method</th><th>Description</th></tr>
|
|||
|
|
<tr><td>/</td><td>GET</td><td>This documentation page</td></tr>
|
|||
|
|
<tr><td>/health</td><td>GET</td><td>Health check with configuration</td></tr>
|
|||
|
|
<tr><td>/marker</td><td>POST</td><td>Convert a file (sync, returns result)</td></tr>
|
|||
|
|
<tr><td>/v1/conversions</td><td>POST</td><td>Convert a file (async-style, returns directly)</td></tr>
|
|||
|
|
<tr><td>/v1/files/convert</td><td>POST</td><td>Convert a file via JSON body with base64</td></tr>
|
|||
|
|
</table>
|
|||
|
|
|
|||
|
|
<h2>POST /marker <small>(multipart/form-data)</small></h2>
|
|||
|
|
<pre><code>curl -X POST http://localhost:8000/marker \
|
|||
|
|
-F "file=@document.pdf" \
|
|||
|
|
-F "force_ocr=false" \
|
|||
|
|
-F "page_range=0,5-10" \
|
|||
|
|
-F "output_format=markdown"</code></pre>
|
|||
|
|
<p>Returns the converted content directly as a file download.</p>
|
|||
|
|
|
|||
|
|
<h2>POST /marker <small>(application/json)</small></h2>
|
|||
|
|
<pre><code>curl -X POST http://localhost:8000/marker \
|
|||
|
|
-H "Content-Type: application/json" \
|
|||
|
|
-d '{
|
|||
|
|
"file_b64": "base64-encoded-file-content",
|
|||
|
|
"filename": "document.pdf",
|
|||
|
|
"output_format": "markdown",
|
|||
|
|
"force_ocr": false
|
|||
|
|
}'</code></pre>
|
|||
|
|
|
|||
|
|
<h2>POST /v1/files/convert <small>(application/json)</small></h2>
|
|||
|
|
<pre><code>curl -X POST http://localhost:8000/v1/files/convert \
|
|||
|
|
-H "Content-Type: application/json" \
|
|||
|
|
-d '{
|
|||
|
|
"file_b64": "base64-encoded-file-content",
|
|||
|
|
"filename": "document.pdf",
|
|||
|
|
"output_format": "json",
|
|||
|
|
"page_range": "0,5-10"
|
|||
|
|
}'</code></pre>
|
|||
|
|
|
|||
|
|
<h3>Options</h3>
|
|||
|
|
<table>
|
|||
|
|
<tr><th>Parameter</th><th>Type</th><th>Default</th><th>Description</th></tr>
|
|||
|
|
<tr><td><b>file</b> / <b>file_b64</b></td><td>file / string</td><td><b>required</b></td><td>The file to convert</td></tr>
|
|||
|
|
<tr><td>force_ocr</td><td>bool</td><td>false</td><td>Force OCR on all pages</td></tr>
|
|||
|
|
<tr><td>paginate_output</td><td>bool</td><td>false</td><td>Separate pages with horizontal rules</td></tr>
|
|||
|
|
<tr><td>output_format</td><td>string</td><td>markdown</td><td>markdown, json, html, chunks</td></tr>
|
|||
|
|
<tr><td>page_range</td><td>string</td><td>all</td><td>Comma-separated page numbers/ranges: "0,5-10"</td></tr>
|
|||
|
|
<tr><td>disable_image_extraction</td><td>bool</td><td>false</td><td>Disable extraction of embedded images</td></tr>
|
|||
|
|
<tr><td>processors</td><td>string</td><td>auto</td><td>Comma-separated full module paths</td></tr>
|
|||
|
|
<tr><td>config_json</td><td>string</td><td>none</td><td>Path to JSON file with additional config</td></tr>
|
|||
|
|
<tr><td>converter_cls</td><td>string</td><td>auto-detected</td><td>Full module path of converter class</td></tr>
|
|||
|
|
<tr><td>use_llm</td><td>bool</td><td>false</td><td>Use an LLM to improve accuracy (requires LLM service)</td></tr>
|
|||
|
|
<tr><td>llm_service</td><td>string</td><td>marker.services.<br>ollama.OllamaService</td><td>LLM service class path: gemini, vertex, claude, openai, azure_openai, ollama</td></tr>
|
|||
|
|
<tr><td>block_correction_prompt</td><td>string</td><td>none</td><td>Custom prompt for LLM block correction</td></tr>
|
|||
|
|
<tr><td>redo_inline_math</td><td>bool</td><td>false</td><td>Re-process inline math with LLM</td></tr>
|
|||
|
|
<tr><td>strip_existing_ocr</td><td>bool</td><td>false</td><td>Remove all existing OCR text and re-OCR</td></tr>
|
|||
|
|
<tr><td>debug</td><td>bool</td><td>false</td><td>Enable debug mode with additional logging</td></tr>
|
|||
|
|
</table>
|
|||
|
|
|
|||
|
|
<h3>Supported Formats</h3>
|
|||
|
|
<p><code>{formats}</code></p>
|
|||
|
|
|
|||
|
|
<h3>Environment Variables</h3>
|
|||
|
|
<table>
|
|||
|
|
<tr><th>Variable</th><th>Default</th><th>Description</th></tr>
|
|||
|
|
<tr><td>OLLAMA_HOST</td><td>http://10.0.1.127:11434</td><td>Ollama instance for OCR fallback</td></tr>
|
|||
|
|
<tr><td>DEESEEK_OCR_MODEL</td><td>deepseek-ocr</td><td>OCR model name in Ollama</td></tr>
|
|||
|
|
<tr><td>AMD_COMPUTE</td><td>false</td><td>Enable AMD ROCm GPU compute (set to "true")</td></tr>
|
|||
|
|
<tr><td>TORCH_DEVICE</td><td>auto</td><td>PyTorch device: rocm, cuda, cpu</td></tr>
|
|||
|
|
<tr><td>MODEL_DTYPE</td><td>float32</td><td>Model dtype: float32, bfloat16</td></tr>
|
|||
|
|
<tr><td>PORT</td><td>8000</td><td>Listening port</td></tr>
|
|||
|
|
<tr><td>HOST</td><td>0.0.0.0</td><td>Listening host</td></tr>
|
|||
|
|
<tr><td>LLM_SERVICE</td><td>marker.services.ollama.OllamaService</td><td>Default LLM service class for use_llm</td></tr>
|
|||
|
|
<tr><td>USE_LLM</td><td>false</td><td>Default use_llm flag (true/false)</td></tr>
|
|||
|
|
<tr><td>OPENAI_BASE_URL</td><td></td><td>Base URL for OpenAI-compatible LLM service</td></tr>
|
|||
|
|
<tr><td>OPENAI_API_KEY</td><td></td><td>API key for OpenAI-compatible LLM service</td></tr>
|
|||
|
|
<tr><td>OPENAI_MODEL</td><td></td><td>Model name for OpenAI-compatible LLM service</td></tr>
|
|||
|
|
</table>
|
|||
|
|
</body></html>Ú/c óR <€ SP R P \ 4 R7 p V # )ú, )Úformats)r2 ÚjoinÚSUPPORTED_DISPLAY)ÚbodyÚ HTML_DOCSs €r* ÚdocsÚcreate_app.<locals>.docsä s% ø€ à× Ñ ¨¯ © Ô2CÓ(DÐ ÓEˆØˆ r, z/healthc ó@ € \ ;' g) \ \ R 4 ' d \ P MRp \ RRR\ R\ R\ RT R \ R
|
|||
|
|
R R \ R
|
|||
|
|
\ R\ ;' g RR\ ;' g R/ 4 # \ d Rp Ldi ; i)r# ÚautoÚunknownÚstatusÚokÚollamaÚ ocr_modelÚ amd_computeÚ torch_deviceÚsupported_formatsÚproviderÚflaskÚdefault_llm_serviceÚdefault_use_llmÚopenai_base_urlNÚ openai_model)r ÚhasattrrE r# rm r r r r r’ rG rH ÚDEFAULT_OPENAI_BASE_URLÚDEFAULT_OPENAI_MODEL)Údevices r* ÚhealthÚcreate_app.<locals>.healthé s« € ð Ü!÷ Hð HÌGÔTcÐey×LzÒLz¤o×&HÒ&Hð AGˆFô Ø �dØ ”kØ Ô*Ø œ;Ø ˜FØ Ô!2Ø ˜Ø !Ô#6Ø œØ Ô 6× >Ð >¸$Ø Ô 0× 8Ð 8°Dð
|
|||
|
|
ó ð øô ô Ø ŠFð ús ‚ B
|
|||
|
|
�)B
|
|||
|
|
Â
|
|||
|
|
|
|||
|
|
B Â B z/markerÚPOST)Úmethodsc ó$ € R \ P 9 Edß \ P R , p V P ;' g R pV P 4 p\ P P RR4 pR\ P P R4 R\ P P RR4 P
|
|||
|
|
4 R8H R\ P P RR4 P
|
|||
|
|
4 R8H RVR\ P P RR4 P
|
|||
|
|
4 R8H R \ P P R 4 R
|
|||
|
|
\ P P R
|
|||
|
|
4 R \ P P R 4 R \ P P R R4 P
|
|||
|
|
4 R8H R
|
|||
|
|
\ P P R
|
|||
|
|
4 R\ P P R4 R\ P P RR4 P
|
|||
|
|
4 R8H R\ P P RR4 P
|
|||
|
|
4 R8H R\ P P RR4 P
|
|||
|
|
4 R8H /p\ W!3/ VB pVR, ' g \ V4 R3# VR8X d3 \ VR, RRRVP R^4 ^ , R2/R7 # \ V4 # \ P ' Edx \ P ! 4 pRV9 g VP R 4 ' g \ R R /4 R 3# \ P ! VR, 4 pVP RR4 pRVP R4 RVP RR 4 RVP RR 4 RVRVP RR 4 R VP R 4 R
|
|||
|
|
VP R
|
|||
|
|
4 R VP R 4 R VP R R 4 R
|
|||
|
|
VP R
|
|||
|
|
4 RVP R4 RVP RR 4 RVP RR 4 RVP RR 4 /p\ W&R , 3/ VB p\ V4 # \ R R!/4 R 3# )"Úfiler9 r: r= r< r Útruer; r? Ú
|
|||
|
|
processorsÚ config_jsonÚ
|
|||
|
|
converter_clsrB rA Úblock_correction_promptÚredo_inline_mathÚstrip_existing_ocrÚdebugr] éô r_ z
|
|||
|
|
text/plainzContent-Dispositionzattachment; filename="r z.md")ÚmimetypeÚheadersÚfile_b64rR r^ ú0JSON body must include 'file_b64' and 'filename'é� FzBNo file provided. Use multipart/form-data or JSON with 'file_b64'.)r ÚfilesrR ÚreadÚformr( ÚlowerrŠ r r
|