Index PDF and Office file contents via Apache Tika
Solr 10 dropped the ExtractingRequestHandler, so a dedicated Tika
container on the Solr VPS (Caddy route /tika/*, own basic-auth
credential; Tika itself has no auth) extracts file text during
indexing: TikaDownloadContentIndexer listens on
BeforeDocumentIsProcessedForIndexingEvent, resolves the file through
the existing DownloadFileResolver and appends the text (mime
whitelist, 30 MB cap, 100k chars, fail-soft) to the document's content
field. Extractions are cached in var/tika-cache keyed on
path+size+mtime - a full re-index of 177 downloads drops from 2:12 min
to 27 s, replacing a file re-extracts naturally. Datasheet
specifications ("625i", "genlock") are now searchable. Spec v1.15.
This commit is contained in:
@@ -3,9 +3,9 @@
|
||||
| | |
|
||||
|---|---|
|
||||
| **Document identifier** | EVO‑VITEC‑HL‑001 |
|
||||
| **Version** | 1.12 |
|
||||
| **Version** | 1.15 |
|
||||
| **Status** | Released |
|
||||
| **Date** | 2026‑08‑24 |
|
||||
| **Date** | 2026‑09‑11 |
|
||||
| **Applies to** | `evomedien/vitec` on TYPO3 v14.3 (headless) |
|
||||
| **Owner** | evomedien — VITEC relaunch |
|
||||
|
||||
@@ -26,6 +26,9 @@
|
||||
| 1.10 | 2026‑08‑18 | **Interface change (additive).** `tx_vitec_domain_model_product` gained five fields: `heroimage` (multiple FAL images, detail payload only), the richtext fields `description2`, `capabilities` and `textrelatedproducts`, and `portfolio` (TCA `link`) — emitted as a **resolved URL** through the new `LinkResolver::typolinkUrl()`. The product payloads are specified for the first time (7.15). Links in `contentelement` / `contentelementcta` that point at a **container** now resolve its children (`items`, page‑level shape) and `background` (7.4). Two record link handlers (`download`, `product`) added to the link browser, resolved server‑side per the new Clause 9.12; new middleware `vitec/download-file` streams `/download/file/<uid>` as a forced download (5.3), file lookup consolidated into `DownloadFileResolver` — first step towards the B‑4 target (10.2). Backend‑only: `relatedprodukt` moved from the Misc tab to General. Editorial: the document footer had been stuck at v1.7 since v1.8. |
|
||||
| 1.11 | 2026‑08‑21 | **New interface: site search.** Apache Solr 10 (dedicated VPS behind an HTTPS reverse proxy) with `apache-solr-for-typo3/solr` 14.0.0-RC1. The EXT:solr results plugin `solr_pi_results` on the search page is rendered headless by `SearchJsonRenderer` (payload key `search`) - request/response contract in the new Clause 7.16. Indexed corpus: pages plus product, market (`detail_page` only), use-case, news and download records; result `type` vocabulary `page\|product\|market\|story\|news\|download`. Downloads gained the canonical route `/download/<slug>` (uid route kept for the record links) and `private_download` is now enforced by `DownloadFileResolver` (5.3, 9.12). The VITEC Set now declares the solr set as a dependency - overriding a foreign set’s TypoScript requires loading after it (5.2). Editorial: the header table had been stuck at v1.9 since v1.10. |
|
||||
| 1.12 | 2026‑08‑24 | **Interface change (additive).** The search endpoint (7.16) gained a type filter and facet counts: request parameter `filter` (a value from the `type` vocabulary; the natural name `type` is unavailable - it is TYPO3’s reserved page-type parameter), response keys `filter` (active filter or null) and `facets.type` (per-type document counts with `active` flags; counts stay complete while a filter is active, except when the filtered result is empty). Editorial baseline of 76 managed synonyms imported into `core_en` (codecs, acquired-brand names such as `exterity => avedia`, UK/US spellings, common misspellings) - synonyms apply at query time, no re-index. New autocomplete endpoint: the EXT:solr suggest plugin as lean JSON page type 7384, deliberately USER_INT (7.16). |
|
||||
| 1.13 | 2026‑09‑11 | **New plugin** `vitec_productfinder` (`ProductFinderJsonRenderer`, payload key `productfinder`): the product category hierarchy on its own, as the data source for the filter selects above the product lists (7.15.3, 8). No change to `vitec_productlist` — filtering happens client side over the products already delivered, keyed on the category uids this payload carries. |
|
||||
| 1.14 | 2026‑09‑11 | **Interface change (additive).** Secondary search facets: stories index their market titles (`market_stringM`, MM relation), products their sys_category titles (`category_stringM`); the search endpoint accepts `market` and `category` GET parameters (values verbatim from `facets.<name>[].value`; an unknown value simply yields zero results) and echoes them as `activeFacets`. Visibility fix: the usecases queue now honours `no_index`/`hideonwebsite`, products defensively `hideonwebsite` - and the excluded story’s document was removed explicitly, because re-indexing never deletes. |
|
||||
| 1.15 | 2026‑09‑11 | **Corpus change (additive, index side).** Download documents now carry the extracted file text: Solr 10 has no ExtractingRequestHandler, so a dedicated Apache Tika container on the Solr VPS (Caddy route `/tika/*`, own basic-auth credential) extracts PDF/Office contents during indexing (`TikaDownloadContentIndexer` on `BeforeDocumentIsProcessedForIndexingEvent`, fail-soft per 9.5, cached in `var/tika-cache` keyed on path+size+mtime). Datasheet specifications are now searchable ("625i", "genlock"). No interface change. |
|
||||
|
||||
This document is drafted in the style of, and adopts the terminology conventions of,
|
||||
ISO/IEC/IEEE 42010 (architecture description), ISO/IEC/IEEE 26514 (information for
|
||||
@@ -904,6 +907,46 @@ rarely shows. The `layout` vocabulary remains `0`–`3` (see the v1.9 note).
|
||||
|
||||
---
|
||||
|
||||
#### 7.15.3 Product Finder payload (`vitec_productfinder`, key `productfinder`)
|
||||
|
||||
The filter above the product lists is fed by its own element. It carries the
|
||||
product **category hierarchy only** — no products, no counts:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "vitec_productfinder",
|
||||
"content": {
|
||||
"productfinder": {
|
||||
"categories": [
|
||||
{
|
||||
"uid": 58,
|
||||
"title": "Platforms and End-Points",
|
||||
"subcategories": [
|
||||
{ "uid": 66, "title": "Avedia Platform" },
|
||||
{ "uid": 67, "title": "EZ TV Platform" },
|
||||
{ "uid": 68, "title": "APEX Platform" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`categories[]` are the children of the configured root category, `subcategories[]`
|
||||
their direct children — the same two levels the product records hang on, so a
|
||||
selected value maps straight onto `product.categories[].uid` in the list payload
|
||||
(7.15.2). The editor places **one** element above the lists; FlexForm offers the
|
||||
category root (empty = the `Product` tree) and the order of both levels (backend
|
||||
order, title, uid).
|
||||
|
||||
Filtering itself is a front-end concern: the list plugin is unchanged, and the
|
||||
front end narrows the products it already received using the URL parameters
|
||||
`?cat=` and `?subcat=`. Categories without products are emitted like any other —
|
||||
the front end decides whether to offer or suppress an empty result.
|
||||
|
||||
---
|
||||
|
||||
### 7.16 Search payload (`solr_pi_results`)
|
||||
|
||||
The site search runs on Apache Solr through EXT:solr. The EXT:solr results plugin
|
||||
@@ -919,6 +962,8 @@ only the rendering differs from the stock Fluid plugin.
|
||||
| `q` | Search terms. Absent or empty: the response keeps its full shape with `numFound: 0`, so the front end never needs a second code path. |
|
||||
| `page` | 1-based result page, optional. |
|
||||
| `filter` | Restrict results to one type from the `type` vocabulary below (e.g. `filter=product`). Unknown values are ignored. Named `filter` because `type` is TYPO3’s reserved page-type parameter. |
|
||||
| `market` | Restrict to one market (stories carry the facet). Value verbatim from `facets.market[].value`; unknown values yield zero results. |
|
||||
| `category` | Restrict to one product category. Value verbatim from `facets.category[].value`. |
|
||||
|
||||
The EXT:solr namespace (`tx_solr[q]`, `tx_solr[page]`) is accepted as a fallback.
|
||||
All three parameters are excluded from cHash validation; the search page is never
|
||||
@@ -936,7 +981,11 @@ identifier).
|
||||
"numFound": 202,
|
||||
"totalPages": 21,
|
||||
"filter": null,
|
||||
"activeFacets": {},
|
||||
"facets": {
|
||||
"market": [
|
||||
{ "value": "Sports, Venues & Entertainment", "count": 9, "active": false }
|
||||
],
|
||||
"type": [
|
||||
{ "value": "news", "count": 137, "active": false },
|
||||
{ "value": "product", "count": 13, "active": false }
|
||||
@@ -963,14 +1012,19 @@ identifier).
|
||||
tabs do not collapse - except when the filtered result is empty, where only
|
||||
the active option (count 0) is returned; front ends should then offer
|
||||
"remove filter" rather than rely on the other counts.
|
||||
- `filter` echoes the active type filter, `null` when none.
|
||||
- `filter` echoes the active type filter, `null` when none; `activeFacets`
|
||||
echoes the active secondary facet filters as `{name: value}`, `{}` when none.
|
||||
- `facets.market` (story market titles, from the MM relation) and
|
||||
`facets.category` (product sys_category titles) appear whenever matching
|
||||
documents carry the fields; further facets are pure TypoScript.
|
||||
- `suggestions` lists spellcheck alternatives ("did you mean"), `[]` if none.
|
||||
|
||||
**Indexed corpus** (`plugin.tx_solr.index.queue`): pages, plus records with a
|
||||
resolvable public URL - products, markets (only those with `detail_page`),
|
||||
use cases, news (`type = 0`; "page as news" records are excluded because their
|
||||
target pages are already indexed) and downloads (`private_download = 0 AND
|
||||
hideonwebsite = 0`). Solutions are not indexed until they carry slugs or detail
|
||||
hideonwebsite = 0`; their `content` additionally carries the file text extracted
|
||||
through Apache Tika, so datasheet specifications are searchable). Solutions are not indexed until they carry slugs or detail
|
||||
pages. Ranking boosts products (^10) and stories (^2). Every executed search is
|
||||
logged to `tx_solr_statistics` with the last two IP octets masked.
|
||||
|
||||
@@ -1005,6 +1059,7 @@ content is extracted from the page’s `tt_content` rows via
|
||||
| CType | TS pattern | Renderer / Processor | Payload key | Kind |
|
||||
|---|---|---|---|---|
|
||||
| `vitec_productlist` | `< lib.contentElementWithHeader` | ProductListJsonRenderer | `products` | list |
|
||||
| `vitec_productfinder` | `< lib.contentElementWithHeader` | ProductFinderJsonRenderer | `productfinder` | taxonomy |
|
||||
| `vitec_productshow` | `< lib.…WithHeader` | ProductShowJsonRenderer | `product` | detail |
|
||||
| `vitec_usecaselist` | `< lib.…WithHeader` | UsecaseListJsonRenderer → **UsecaseSerializer** | `usecases` | list |
|
||||
| `vitec_usecaseshow` | `< lib.…WithHeader` | UsecaseShowJsonRenderer → **UsecaseSerializer** | `usecase` | detail |
|
||||
@@ -1368,4 +1423,4 @@ remediation.
|
||||
- Header convention — the uniform header section across CEs, plugins and containers.
|
||||
- `Configuration/Sets/Vitecset/setup.typoscript` — the single TypoScript entry point.
|
||||
|
||||
*End of document EVO‑VITEC‑HL‑001 v1.12.*
|
||||
*End of document EVO‑VITEC‑HL‑001 v1.15.*
|
||||
|
||||
Reference in New Issue
Block a user