01 A little introduction
Rujikorn Ngoensaard
Senior Data Engineer
I build and operate
enterprise data platforms.
Where I started
Bachelor’s in Electrical and Electronic Engineering,
Ubon Ratchathani University, 2013–2017.
Outside of work
02 The journey
An evolving practice across manufacturing, commerce and enterprise data.
2013–2017 · The foundation
Electrical and Electronic Engineering
Ubon Ratchathani University
Nov 2017–May 2020
Process Engineer, Engineer I
Developed diagnostic and repair-routing automation for HDD testing. Operated scripts across 130+ Linux VMs and coordinated technical repair and retest work.
May 2020–Dec 2021
Product Design Engineer
Promoted from Senior to Staff Engineer
Automated reporting and centralized data workflows with Airflow. Redesigned ML prediction services for HDD testers.
Jan 2022–Jan 2023
Data Engineer
Built personalization pipelines, search and product-discovery services. Selected website placements recorded more than 50% relative improvement in click-through rate.
Feb 2023–Jul 2024
Data Engineering Specialist
Delivered AWS lakehouse pipelines for 150+ sources within assigned domains. Built governance foundations and enabled experiment tracking through a private MLflow deployment.
Oct 2024–Present
Senior Data Engineer
Build and operate enterprise data platforms across integration, analytics, governance and production support. Lead technical delivery with three outsourced engineers and design target architecture with infrastructure and security stakeholders.
03 Selected work
Senior Data Engineer
October 2024–Present
I build and operate enterprise data platforms and lead technical delivery with three outsourced engineers.
Traced lineage and coordinated Tableau-to-Fabric delivery. Delivered in three months.
Built scheduled daily access processing. Rolled out July 2026.
Assessed gaps and designed the target architecture. Direction approved; preparation ongoing.
Build and validate enterprise ingestion with ADF, Databricks and ADLS.
Developed naming, ownership and access standards; adoption continues.
Investigate platform constraints and coordinate incidents, capacity decisions and vendor follow-up.
Analytics Migration Central Pattana
Project delivered in three months:
Move Tableau analytics to Microsoft Fabric and Power BI.
Missing dictionaries, specifications and source ownership required dashboard-to-source lineage reconstruction.
I led technical delivery and coordinated data engineering with the project team.
Direct source-database access was restricted for this migration. Source delivery used data files plus accompanying metadata files for file-level validation.
Source delivery
ADF
Zero records Skip
Skip all remaining downstream tasks for that batch.
Nonempty batch Databricks
Check row counts and columns against the accompanying metadata.
Power BI results must match the equivalent Tableau report for the same reporting period.
RLS Automation Central Pattana
Rolled out July 2026: daily processing and 12 domain workbooks.
Replace person-operated refresh, extraction and Excel macros with scheduled daily processing.
Preserve established access decisions and downstream compatibility.
I owned engineering for the master RLS pipeline and dashboard distribution.
Organisation data + prior approvals
Scheduled ADF/Databricks processing
Workbooks + Fabric datasets
New groups
Business decisions
Reviewed assignments feed a later run
Current organisation data identifies who exists; approved assignments and business review determine access. Regular users can inherit their existing organisation group’s approved assignment.
No.A reviewed No is a decision; blank is not approval. Groups absent from the approved baseline enter the review queue with blank review fields and effective access of No while unresolved. Daily processing continues with outstanding reviews.
I import existing business decisions before regenerating the queue, apply completed assignments, retain unresolved items and remove resolved items from the next queue. Later business changes feed a subsequent run.
Fabric Lakehouse supplies current organisation and user data. SharePoint supplies the prior approved master and business review workbook; lookup and dashboard configuration define access attributes and distribution.
Bronze, Silver and Gold processing produces a master workbook with Group, User and Special User sheets, one new-group review workbook, 12 domain-specific dashboard workbooks and ReportRLS datasets for configured Fabric destinations. Regular-user records carry 13 business-rule-driven access attributes.
Dashboard outputs are fresh .xlsx files; generation does not run VBA or require existing macro-enabled workbooks. Monthly workbook naming does not change the daily processing cadence.
Acceptance concerns approved legacy macro outputs and business rules: sheet structures, headers, column order, assignments, all 13 regular-user attributes, required blanks and dashboard mappings.
Special users retain approved exception assignments and values. They do not inherit regular organisation-group assignments or have the 13 regular-user attributes recalculated. They are appended to regular users without deduplicating by email, preserving the required legacy behavior (UNION ALL).
Configured PII is encrypted before persistence in Databricks data. Deterministic hashes support identifier matching across protected datasets; Azure Key Vault protects encryption material. DEV and PRD use separate security configurations and key material.
Authorised export processing decrypts required PII. Excel outputs and configured Fabric destinations may contain decrypted information; logs must exclude plaintext employee data and secrets.
ADF orchestrates separate Databricks activities. Pipeline run identifiers connect processing and structured logs. Validation covers source structure, row counts, protected joins, assignments and output schemas.
Source, permission, processing and publishing errors surface as failures. Publishing handles locked SharePoint files. Setup and deployment operations are separate from daily processing; business reviewers retain access decisions and approval.
Azure Target Architecture Central Pattana
Target direction approved; transition preparation ongoing; publication method open. Implementation and recovery validation remain incomplete.
Address ownership, access, publication and operational-readiness gaps while retaining working controls.
Responsibilities and controls were inconsistent across platform and organisational boundaries.
I assessed the platform, designed the target and aligned responsibilities and transition priorities with infrastructure and security stakeholders.
CPN
Public paths
Daily OneLake copies
RIS foundation / CPN workloads
Private paths / Approved destinations
Publication undecided
Separate approvals
Existing: Unity Catalog, DEV/PRD. Fabric access stays separate.
Proposed: recovery objectives, restoration tests, release approval, tested rollback.
Solid: current. Dashed: proposed.
I assessed the platform using confirmed interview answers. The assessment is a draft with content review Pending, not an independently verified audit. Its findings shaped the target architecture and transition priorities with infrastructure and security teams.
Retain Unity Catalog-governed ADLS access, DEV/PRD separation, centralized secrets, production failure alerts, cost controls and unchanged DEV-tested artifact promotion. Existing account-removal and integration practices remain. Unity Catalog does not enforce Fabric access.
| Assessment theme | Finding → proposed response |
|---|---|
| Ownership and access | Responsibilities are undocumented and catalog owners unassigned. I proposed accountable data ownership, recorded approvals, periodic reviews and time-limited access. |
| Controlled connectivity | Partial network restrictions informed private service paths, shared network responsibilities and approved outbound destinations, distinguishing browser access from data transfer. |
| Trusted publication | Daily copies lack ownership and validation. The response pairs publication ownership and consumer approval with freshness checks and source-to-copy reconciliation. |
| Operational readiness | Alerts and artifact promotion already work. Missing recovery and release safeguards call for agreed recovery objectives, restoration testing, release approval and tested rollback. |
Establish the baseline, assess gaps, define the target and operating model, then sequence and validate change. This describes the method, not completion of all stages. Architecture enables improvements; implemented, tested practices demonstrate closure.
Team Head approved the target direction; individual standards require separate approval. RIS owns identity and shared foundation; CPN owns delegated platform workloads. Domain workspaces and catalog ownership remain proposed.
Current Databricks-to-OneLake publication uses daily copies over the public internet. The future publication method and connectivity remain open. Fabric consumer access requires separate approval.
Transition preparation is in progress. Migration, control closure, recovery readiness and measured risk reduction are not complete. Restoration testing and release rollback remain separate obligations.