Backend module: CSV import for the VITEC domain models
New "VITEC Import" module under Web: one import page per domain model (v1: Market, Solution, Product - registry-driven, adding a model is one config entry). Workflow: upload a CSV (delimiter and encoding are auto-detected, including German-Excel semicolon/Windows-1252), map CSV columns to DB fields, persist the mapping per model together with the identity field used for matching (new table tx_vitec_import_mapping, no TCA - pure tool configuration), review a unified list of CSV rows matched against the DB records (new / update with differing fields / unchanged / db-only), then apply the checked rows through DataHandler. Each importable row carries an editable JSON payload textarea - what is written is the textarea content, not the raw CSV, so editors can fix values right in the review step. The parsed CSV travels through the form as a hidden JSON field: no session state, no temp files. Importable fields are derived from TCA at runtime (scalar types only; files, categories and other relations are excluded - a flat CSV cannot carry them). Payloads are whitelisted against that field list on apply; new records require a storage pid (prefilled from existing records). BE user permissions apply via DataHandler. Deliberately out of v1: import log with three-way compare (protection against overwriting manual edits), images/relations, multiple saved mappings per model. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
baa0fdd331
commit
10c0328bf2
9 files changed
+831
No files matched your search
@@ -0,0 +1,78 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace Evomedien\Vitec\Import;
|
||||
|
||||
/**
|
||||
* Tolerant CSV parser for editor-supplied files.
|
||||
*
|
||||
* German Excel exports ship with semicolon delimiters and Windows-1252
|
||||
* encoding; other tools use comma/UTF-8. Both are detected instead of
|
||||
* configured: encoding by BOM/UTF-8 validity, delimiter by counting
|
||||
* candidates in the header line. Multiline quoted values (e.g. RTE
|
||||
* descriptions) are handled by fgetcsv on a temp stream.
|
||||
*/
|
||||
final class CsvReader
|
||||
{
|
||||
/**
|
||||
* @return array{columns: array<int,string>, rows: array<int,array<string,string>>}
|
||||
*/
|
||||
public function parse(string $content): array
|
||||
{
|
||||
if (str_starts_with($content, "\xEF\xBB\xBF")) {
|
||||
$content = substr($content, 3);
|
||||
}
|
||||
if (!mb_check_encoding($content, 'UTF-8')) {
|
||||
$content = mb_convert_encoding($content, 'UTF-8', 'Windows-1252');
|
||||
}
|
||||
|
||||
$firstLine = strtok($content, "\r\n") ?: '';
|
||||
$delimiter = ';';
|
||||
$best = 0;
|
||||
foreach ([';', ',', "\t"] as $candidate) {
|
||||
$count = substr_count($firstLine, $candidate);
|
||||
if ($count > $best) {
|
||||
$best = $count;
|
||||
$delimiter = $candidate;
|
||||
}
|
||||
}
|
||||
|
||||
$stream = fopen('php://temp', 'r+');
|
||||
fwrite($stream, $content);
|
||||
rewind($stream);
|
||||
|
||||
$header = fgetcsv($stream, null, $delimiter, '"', '');
|
||||
if (!is_array($header)) {
|
||||
fclose($stream);
|
||||
return ['columns' => [], 'rows' => []];
|
||||
}
|
||||
$columns = [];
|
||||
foreach ($header as $i => $name) {
|
||||
$name = trim((string)$name);
|
||||
$columns[$i] = $name !== '' ? $name : ('column_' . ($i + 1));
|
||||
}
|
||||
|
||||
$rows = [];
|
||||
while (($data = fgetcsv($stream, null, $delimiter, '"', '')) !== false) {
|
||||
if ($data === [null] || $data === ['']) {
|
||||
continue; // blank line
|
||||
}
|
||||
$row = [];
|
||||
$hasContent = false;
|
||||
foreach ($columns as $i => $name) {
|
||||
$value = trim((string)($data[$i] ?? ''));
|
||||
$row[$name] = $value;
|
||||
if ($value !== '') {
|
||||
$hasContent = true;
|
||||
}
|
||||
}
|
||||
if ($hasContent) {
|
||||
$rows[] = $row;
|
||||
}
|
||||
}
|
||||
fclose($stream);
|
||||
|
||||
return ['columns' => array_values($columns), 'rows' => $rows];
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user