Skip to main content
FromNine
Menu

AI & data

Generative AI built for daily use, not for a demonstration.

We engineer generative AI on the documents and data your organisation already holds. Retrieval follows the access rights of each source, every release must pass an agreed test set, and the cost and response time of each answer are on a dashboard.

For Malaysian organisations that want generative AI on their own knowledge, with personal data treated as Malaysian data protection law expects.

Arrange a discussionEnterprise knowledge
AI Engineering & GenAI: the four layersA stack of four layers, read from top to bottom: retrieval from the sources you choose, a gateway that routes each request to a model, a layer of tests and guardrails, and the application in which your staff work. A single path leads from the sources up to the application.01Retrieval from chosen sources02Gateway that routes to models03Tests and guardrails04Live application
  1. 01Answers point to the passage they rely on, and say plainly when no passage supports them
  2. 02A test set and release thresholds settled with your specialists before launch
  3. 03Per-request budgets, so finance sees what each answer costs and how long it takes

01 Common problems

Why generative AI projects stall

Choosing a model is seldom the difficult step. Projects stall on the data, controls and operations around it.

  1. 01

    The demonstration went well, but quality was never measured

    A handful of colleagues tried a handful of questions and declared it good. Without a test set, any change to a prompt or a model version is a guess.

  2. 02

    Staff can see documents meant for others

    Files went into the index stripped of their permissions. Information security and your DPO pause the rollout, with good reason.

  3. 03

    Spending rises faster than adoption

    Each question is sent to the biggest model with as much context as possible. Nobody can say what a single answer costs, so nobody controls the bill.

  4. 04

    Answers read well and are wrong

    Search brings back passages that almost match, the model fills in what is missing with confidence, and users stop trusting it after a few errors.

02 Scope of the service

What the service covers

  1. 01 Retrieval and knowledge systems

    We design retrieval for each source separately: how its content is parsed and split, how keyword and vector search are combined, and how its access rights stay attached to every passage. This underpins assistants over your enterprise knowledge.

    Activities

    • Inventory of sources with their owners, formats and permission models
    • Rules for splitting and tagging, by document type
    • Keyword and vector search combined, with a re-ranking step
    • Search results filtered by the rights each source system grants

    What you would receive

    • Pipelines that detect new or edited content and re-ingest it
    • Retrieval service that quotes its passages and declines when evidence is thin
    • Retrieval scores reported source by source

  2. 02 Models you can swap without rebuilding

    Your applications talk to one gateway, not to a model vendor. The gateway decides per task which model answers, fails over when a provider is down, and makes a later change of model a configuration matter.

    Activities

    • Choice between hosted frontier models, deployments in a region you select and open-weight models you run yourself
    • Routing rules based on the task, the sensitivity of the data and the cost
    • Fallback models, time limits, caching and rate limits
    • Each provider’s terms on data use checked with your legal and security teams

    What you would receive

    • A gateway whose routing rules are configuration, not code
    • A record of the model decision for each use case
    • Spending and latency ceilings set per request category

  3. 03 Testing generative AI like any other software

    The people who own the business process agree what a good answer looks like, and we turn that into test sets that run with every change. Before automated grading is trusted, we compare it with human reviewers.

    Activities

    • Reference question sets per use case, prepared with subject matter experts
    • Separate offline tests for retrieval and for generated answers
    • Attack testing aimed at prompt injection and data leaks
    • Tests in production using user feedback and sampled reviews

    What you would receive

    • A test suite that blocks a release in the CI pipeline when quality drops
    • Acceptance criteria signed by the accountable business owner
    • Findings from adversarial testing, with the mitigations applied

  4. 04 LLMOps and monitoring

    One trace follows a request from the user’s question to the retrieved passages, the model call and the final answer. Prompt and model versions are managed like source code, and production enforces each budget.

    Activities

    • Tracing and structured logs for each request
    • Version control for prompts, models and search indexes
    • Caching of exact and similar questions where this is safe
    • Monitoring of drift and quality, with alerts

    What you would receive

    • Dashboards showing quality, spending and response time
    • Playbook for incidents that only AI systems produce
    • Step-by-step guide for refreshing a model or an index

  5. 05 Personal data and governance from the start

    Training and retrieval data often contain personal data, so we design for the Personal Data Protection Act 2010 (Act 709), the PDPA, as amended by Act A1727: purpose, notice and the Security Principle. Where output is used in the EU, we also prepare the logging, oversight and documentation the EU AI Act may require. Your counsel makes the legal assessment.

    Activities

    • Classification of each use case and of your role under the EU AI Act, where it applies
    • Checks of personal data in sources and prompts against the PDPA, and input to GDPR impact assessments for EU use
    • Design of logging and of oversight by people
    • Model and system documentation stored alongside the code

    What you would receive

    • A documentation pack for the system
    • A design for logs and how long they are kept
    • Written procedure naming who oversees each use case

  6. 06 Fine-tuning and document understanding, when tests justify them

    Fine-tuning, distillation and multimodal document reading each fix a narrow problem. They come into play after test results prove that better retrieval and prompting still fall short of the agreed bar.

    Activities

    • Analysis of gaps based on test results
    • Plan for preparing and labelling any training data
    • Distillation into smaller models to lower cost or response time
    • Layout-aware extraction from scanned and complex documents

    What you would receive

    • Side-by-side test scores, before and after
    • A record of where training data came from
    • A decision to keep, reverse or withdraw the change

03 Architecture

A reference architecture, for illustration

Most generative AI systems we deliver for organisations have the same outline. Retrieval draws on your sources, a gateway chooses the model, guardrails check what goes into and comes out of the model, and a loop of tests and monitoring decides which version is allowed into production.

Permissions are checked inside retrieval rather than in the screen the user sees. A passage from a document that someone may not open in the source system is never sent to the model for that person.

Layers shown

Sources
Document repositories, wikis and business applications, plus the identity provider that holds each person’s access rights.
Retrieval
Ingestion into a combined index, re-ranking, and the permission check.
Models
A gateway that chooses between hosted, regional and self-run models, with fallbacks and caching.
Application
Answers that cite sources, a refusal when evidence is lacking, and checks on what goes in and comes out.
Operate and improve
Reference sets, release thresholds, traces and budgets; today’s production traces become tomorrow’s tests.
Illustrative example

  1. Sources

    • Document repositories and wikis
    • Business applications
    • Identity provider

  2. Retrieval

    • Ingestion and splitting
    • Keyword and vector index
    • Permission check
    • Retrieval with re-ranking

  3. Models

    • Gateway to models
    • Hosted, regional and self-run models
    • Guardrails on input and output

  4. Application

    • Application that cites sources

  5. Operate and improve

    • Reference sets and release thresholds
    • Traces, cost and response-time budgets
Retrieval, gateway and the testing loop

For illustration only; this is not a system built for a client.

Read the diagram as text

Content travels from document repositories into ingestion, then into a combined keyword and vector index, through retrieval and re-ranking, and via the model gateway into the application.

Business applications feed ingestion too. Your identity provider drives a permission check that narrows what retrieval may return. Guardrails inspect traffic into and out of the application, and the gateway can call hosted frontier models, regional deployments or open-weight models running on your own infrastructure.

Underneath, reference sets and release thresholds form the test suite, while monitoring keeps traces and tracks cost and response-time budgets. Production traces flow back into the test suite.

04 Trade-offs and limits

Trade-offs we plan for

  • No quality claim without a measurement

    What we do about it

    Your process owners help set the quality criteria, and we test the AI output against them ahead of each release and again afterwards.

    We assess retrieval and answer generation one at a time, so a fault can be traced to its source.

    Limits and prerequisites

    Generated output is sometimes wrong. For that reason uncertain cases go to a reviewer, and the system declines when the evidence is missing.

    A test set reflects the knowledge of the experts behind it. We need some of their time at the start.

  • Access rights stay attached to content

    What we do about it

    Permissions are recorded when content is ingested and checked again against your identity provider each time someone asks a question.

    Limits and prerequisites

    When a source system keeps its permissions in a form we cannot interpret reliably, that content waits outside the index until the issue is fixed.

    Content shared too widely in the source remains shared too widely. Search makes it easier to find, so tidying the source sometimes comes first.

  • Cost and response time are part of the design

    What we do about it

    Every type of request receives a budget. Routing, caching and the amount of context are adjusted to stay within it, and the dashboards show where spending goes.

    Limits and prerequisites

    Price lists and rate limits sit with the provider. The gateway lets you react quickly, though a higher price stays higher.

  • Models change without notice

    What we do about it

    We fix each model to a specific version, and any upgrade must pass the complete test suite before users see it.

    Limits and prerequisites

    Providers occasionally withdraw hosted models at short notice. We prepare an alternative for each use case, which the business owner must approve.

05 Working method

Our working method

Sequence matters to us: the way to measure is settled before the build begins.

  1. 01

    Define the use case

    Users, the decisions supported, the sources permitted and the price of a wrong answer, all agreed in writing.

    ResultBrief and risk class for the use case

  2. 02

    Assemble the reference set

    Your subject experts supply real questions with expected answers, including some the system ought to refuse.

    ResultTest set under version control

  3. 03

    Get retrieval right first

    Ingestion, index and permission check, adjusted until retrieval quality reaches the agreed threshold.

    ResultReport on retrieval quality

  4. 04

    Release only past the thresholds

    Generation, guardrails and the gateway reach production only after the CI test suite turns green.

    ResultRelease with test evidence

  5. 05

    Run and refine

    Traces, sampled reviews and user feedback shape the next test set and the next release.

    ResultMonthly figures on quality, spending and latency

06 Human oversight

Where decisions stay with people

The model drafts, searches and suggests. Your people set what it may read, define a good answer and choose what happens when it hesitates.

  • Business owners approve quality

    The business owner accountable for each use case signs the quality criteria and the release evidence; the project team cannot do so alone.

  • Doubtful answers reach a person

    When confidence falls below an agreed level, the system declines or passes the question to a named team.

  • Permissions are set at the source

    Read access to a document is decided in its source system and your identity provider. The AI layer never grants it.

  • User feedback improves the system

    Users can mark answers as wrong. People review each mark, and confirmed errors are added to the test set.

07 Technology

Technology we use

Retrieval
  • OpenSearch and Elasticsearch
  • PostgreSQL with pgvector
  • Azure AI Search
  • Cross-encoder re-rankers
Models
  • Hosted frontier models
  • Regional managed model services
  • Open-weight models on vLLM
  • Embedding models suited to each language
Testing and operations
  • OpenTelemetry tracing
  • Test suites that run in CI
  • Registries for prompts and models
  • Langfuse or a comparable tool
Application
  • Services in TypeScript and Python
  • Streaming APIs
  • Existing portals and the front ends of SAP, Odoo and Salesforce

These names reflect hands-on engineering work. Their inclusion does not mean that a vendor partners with or endorses FromNine.

08 Scenario

An illustrative scenario

Illustrative example

A compliance assistant for a capital market intermediary

01Starting point
Compliance staff at a capital market intermediary check daily questions against the Guidelines on Technology Risk Management of the Securities Commission Malaysia (SC) and the firm’s own procedures. Answers vary between colleagues, and finding the relevant paragraph takes time.
02What we would deliver
Retrieval over the SC guidelines and the internal procedures, with access rights taken from the document repository, and answers that quote the paragraph they rely on.
03Decisions that stay with people
The head of compliance approves the test set and every release that changes answers; questions outside the documents are declined and passed to a named officer.
04How results would be tracked
Share of answers citing the correct paragraph, the rate at which out-of-scope questions are declined, and the cost of each answer.

09 By industry

By industry

  • Public Sector & Government

    Assistants for public officers and the public that cite the governing rule, can answer in Malay and English where a service needs both, and record what was retrieved for whom.

  • Financial Services

    Assistants over policies and procedures for banks, insurers and takaful operators, with tight entitlements, full traceability and model risk documents your second line can examine.

  • Healthcare & Life Sciences

    Search across clinical protocols and product files within clear limits: no clinical decisions, and sensitive personal data kept in environments you have approved.

Malaysia

Personal data in prompts and sources

Retrieval sources, prompts and training sets often hold personal data. In Malaysia that brings the Personal Data Protection Act 2010 (Act 709), the PDPA, into play, as amended by Act A1727, whose changes came into operation in three stages in 2025. The data controller must respect purpose and notice, processors are bound by the Security Principle, and since 1 June 2025 a personal data breach must be notified to the Personal Data Protection Commissioner, within 72 hours under the Commissioner’s guidance.

The National Guidelines on AI Governance and Ethics (AIGE) are voluntary, and the National Artificial Intelligence Office (NAIO) was launched under the Ministry of Digital on 12 December 2024. Where your AI output is used in the EU, the EU AI Act can apply as well. We design retrieval, logging and documentation once, so they answer the Malaysian questions and the European ones.

Questions on AI engineering

Which model would you advise?

That follows from the task, the data it touches and where the system may be hosted. In practice we combine models: a strong model for steps that need reasoning, smaller or self-run models for classifying and extracting, and a deployment in a specific region where the hosting location agreed with you and recorded in the contract requires it.

Because the gateway keeps the choice open, the advice can change when models and prices do.

What level of accuracy is realistic?

No percentage is promised until your own data has been tested. We fix the quality criteria together, build the test set and publish results against it ahead of launch and after each change.

Every model makes mistakes, so each design states what follows an error: the system declines, a reviewer checks, or the question goes to a member of staff.

Might our data be used to train a third party’s model?

Not in systems we design. Either the model service’s contract excludes training on your inputs, or the model runs inside your own environment; your security team and we document which applies, use case by use case.

Could the EU AI Act cover a Malaysian organisation?

It can. The Act covers providers placing AI systems on the EU market wherever they are based, and providers and deployers outside the EU when the system’s output is used in the EU. We map which obligations may follow from your use case and role, design logging, oversight and documentation to suit, and leave the conclusion to your counsel. In Malaysia, the National Guidelines on AI Governance and Ethics (AIGE) are voluntary guidance, not law.

How do you protect sensitive and personal data?

FromNine is ISO/IEC 27001 certified. Before we build, we agree access boundaries, the hosting location and retention with you, including how personal data is treated under the PDPA. Adversarial tests for prompt injection and data leakage are a fixed step in the release process.

Where should we begin?

Pick one use case with a named owner, a few well-understood sources and a test set your experts help write. The result is a running system plus measured quality, cost and response time, which is what the next funding decision should rest on.

Bring one use case and the questions that worry you most

Our first step is to show you how it would be tested; building comes after that.

Arrange a discussionEnterprise knowledge