hamsa software
← All articles
Automation6 min read

What the CSV importer thought you meant

A five-row price file reveals the gap between parsing text and agreeing what changes it authorises.

A triangle receives one line and releases several diverging lines, suggesting a parsed file separated into proposed and rejected changes.

Successful parsing is an early milestone

The price file can parse successfully after product 00124 has already become 124. That is a peculiar kind of success if the zeros are part of the catalogue identifier. Expected columns and plausible amounts do not tell the uploader whether the importer matched the right product or chose the right change. I would want the preview to show what it understood and what it plans to do, while there is still time to disagree without changing live prices.

Take a deliberately small price-update file. Its columns are product_code, description and price_gbp. Five data rows include a quoted description, a missing amount, two competing prices for one product, one addition and one update. The example uses an invented catalogue with three existing products. It is a snapshot of proposed prices for the listed products: an omitted product keeps its current price. It is neither a replacement catalogue nor a stream of increments. Those choices belong in the import contract because the same CSV syntax could serve any of those meanings.

The accompanying Python program was run on this small file to produce a preview, with no catalogue writes. Three rows were rejected and two changes were otherwise valid; the declared all-or-nothing policy blocked the whole batch. That lets us inspect the product matches, blank amount and competing prices before deciding whether any change should proceed. An upload-complete message would answer an earlier question about receiving the file, not those later questions about what the file authorises.

Preserve the tokens before interpreting them

The first row contains 00124, “Filter, fine” and 12.50. The comma inside the quoted description belongs to the description, not the column boundary. Python's CSV reader handles quoting and normally returns fields as strings; its documentation also explains why file readers should use the appropriate newline handling.[1] Splitting each line on commas or converting every numeric-looking token immediately would damage this example. The fixture's actual parsed values retain the leading zeros and the complete description. The raw CSV is included in the downloadable script so the interpretation can be inspected.

Amounts need a more opinionated interpretation than product codes. Here the file contract permits a non-negative GBP value with a decimal point and exactly two decimal places. The accepted text becomes a Decimal without going through a binary float.[2] That is a deliberate grammar for this example, not a prediction of every UK spreadsheet export. It rejects blanks, currency symbols and comma-formatted amounts. A more accommodating importer can document other spellings, but should still show its interpretation rather than guess at decimal punctuation. Explicit zero is allowed; an empty token has supplied no such amount.

The identifiers receive a different interpretation. Product codes remain text, including leading zeros. No universal trimming or case-folding rule is assumed. If an existing catalogue treats a trailing space as insignificant, the preview should show the original and matched code when that normalisation occurs. Otherwise two visually similar inputs can silently merge. Before applying row-level rules in a production importer, also validate the expected headers, row widths, encoding and size limits. The small fixture assumes its published header and shape; it is a worked price preview, not a general-purpose hostile-file parser.

Keeping these layers separate makes errors easier to explain. A malformed quoted field is a parsing problem. An unsupported amount spelling is a conversion problem. An unknown product might be an allowed addition or a business rejection, depending on the agreed purpose. Calling all three Invalid CSV gives the uploader no useful repair. It can also tempt an engineer to loosen the parser when the actual disagreement concerns whether the file may create products. A preview should explain the relevant layer using the supplied row and token.

Locate the failed interpretation before changing the importer

Three distinct checks in the five-row price-file example.

Follow the labelled arrows. On a narrow screen, scroll sideways if needed.

Locate the failed interpretation before changing the importerParse the declared CSV grammar. Preserve the quoted comma in Filter, fine as one field. Convert under the amount contract. 00420 has a blank price: reject under the explicit non-negative, two-decimal GBP contract. Resolve the business conflict. 00124 proposes both 12.50 and 13.00. Neither competing snapshot value wins automatically. From Parse the declared CSV grammar to Convert under the amount contract: Parsed fields still need conversion. From Convert under the amount contract to Resolve the business conflict: Valid amounts still need policy.Parsed fieldsstill needconversionValid amountsstill needpolicyParse the declared CSVgrammarPreserve the quoted comma inFilter, fine as one field.Convert under theamount contract00420 has a blank price:reject under the explicitnon-negative, two-decimal GBPcontract.Resolve the businessconflict00124 proposes both 12.50 and13.00. Neither competingsnapshot value winsautomatically.
Loosening CSV parsing cannot resolve a disagreement about which product price the file authorises.

A preview should show decisions, including decisions to stop

Both 12.50 and 13.00 claim to be the new price for 00124. The preview keeps its existing 11.00 visible and rejects both proposals; the last line does not win merely by being nearer the end of the file. That follows from treating the upload as a snapshot of desired prices. A dated sequence of price-change events would tell another story, in which repeated product codes might be legitimate successive changes. The business meaning of the file decides which interpretation is right.

Product 00420 has an existing price of 3.00 and a blank proposed amount. It is rejected; the preview does not replace the price with zero or infer Keep current without a stated rule. Product 00555, Brush, has no existing entry and proposes an addition at 9.99. Product 00660, Cloth, proposes an update from 19.99 to 21.00. These last two are valid candidates, not completed changes. A summary saying Two products imported would be false under the chosen batch policy. Two changes proposed; batch blocked gives the operator a usable account of the work still required.

Blocking a whole file is not always kinder to the work. Thousands of independent stock updates might reasonably proceed while one malformed description waits for repair, provided the operator gets an exact ledger of accepted and rejected rows. The next upload then becomes harder to explain: some work is already complete. A price list intended to take effect coherently may be easier to manage as one blocked batch. I would choose between those outcomes by asking whether mixed old and new prices are acceptable, rather than inheriting the import library’s habit of processing one row at a time.

The preview must represent the complete decision set even if the screen only shows a sample. A count of duplicate keys and a downloadable or filterable rejected-row view can expose a problem outside the first ten lines. Line numbers in this fixture are straightforward because each record occupies one physical line. With quoted multiline fields, distinguish record numbers from physical line positions so the person repairing the file can find the right content. A useful error is an instruction someone can locate in their source, not merely a technically correct objection stored in a server log.

Five parsed rows, no catalogue writes

Executed preview over an invented price list. The declared batch policy blocks all changes while any row is rejected.

Scroll sideways to see all columns.

Line / codeExisting GBPInput GBPPreview decision
2 / 0012411.0012.50Reject: duplicate code
3 / 004203.00blankReject: explicit amount required
4 / 0012411.0013.00Reject: duplicate code
5 / 00555Absent9.99Add proposed; batch blocked
6 / 0066019.9921.00Update proposed; batch blocked
The parser preserved 00124 and the quoted description Filter, fine. Parsing succeeded; duplicate-key and missing-value policies blocked execution.

Re-uploading should have a declared meaning

For this listed-product price snapshot, repeating a successfully applied, unchanged file against the same unchanged catalogue should produce unchanged rows. It must not add 9.99 to an existing amount or create another product because the uploader pressed the button twice. A corrected file should receive a new import identity and a link to the earlier attempt, rather than rewriting the rejected record into a claim that it was always valid. This is a semantic choice about replacement values, separate from any request-level duplicate protection on the upload endpoint.

A file digest helps identify identical bytes, but it does not establish an identical change plan. The catalogue may change after preview, or someone may alter the column mapping or conversion rules. Preserve the interpreted preview and check the relevant assumptions when applying it. That requires enough before-and-proposed information for the later comparison to mean something. Calling the file prices-final.csv is useful for finding it; unfortunately, the filename cannot prevent either a catalogue edit or the arrival of prices-final-again.csv.

For a small, rarely changed catalogue, a read-only report and explicit transactional apply step may be enough; a whole import portal could add more upkeep than the task warrants. The report still has to reveal the decisions this file exposed: the quoted comma stays inside a description, the zeros identify a product and conflicting prices stop this batch. Seeing those interpretations before committing is what gives the uploader a real chance to correct them. A thumbnail of five successfully uploaded rows does not offer the same chance.

Working on a workflow like this? Hamsa helps finance and operations teams connect their tools and handle the exceptions, starting with one workflow.

See the Payment Operations Pilot

References & further reading

Primary sources behind this article. Living documentation may change.

  1. Python: CSV parsing and dictionary readers
  2. Python: Decimal construction from strings
Hamsa Jama

Practical engineering. A human perspective.

More about Hamsa Software and the person behind it. Editorial notes.

A few more things
worth a read.

Back to the reading room
Automation6 min read

A budget that three workers can share

Enforce task limits through atomic reservations and repeat-safe settlement, including the uncomfortable case where a timed-out operation may still be spending.

Automation7 min read

What exactly did that approval authorise?

Bind an assistant’s approval to the action, target and material revision, then enforce that binding when the write happens.