Skip to main content

SubForm and GC Template Flow

Legend​

ElementsGMS / GMS1Notes
SourceOfFundBudget SourceSame funding source concept across both applications.

Terminology notes:

  • Ingestion means parsing the Excel template, validating fields using FormMaster/FormMasterFields rules, and preparing/saving FormSubmissionData.
  • Downstream handoff means the PostDTA upload path calling GMS2 process action and publishing the MQ event to TPGMS (GMS1).

High-level Flow​

flowchart TD
    A[User uploads Excel] --> B{Endpoint}
    B -->|/elements/subform| C[SubForm flow]
    B -->|/elements/postdtasubform| D[GC template flow]

    C --> C1[Load IVP and scheme config]
    C1 --> C2{HasSubFormIngestion / family rules}
    C2 -->|true| C3[Validate scheme from Excel]
    C2 -->|false| C4[Upload file only]
    C3 --> C5[IngestForm]
    C5 --> C6[SaveIngestedForm]
    C6 --> C7[Update post-ingestion state]

    D --> D1[Load IVP and scheme config]
    D1 --> D2{Family = TaxCredit}
    D2 -->|true| D3[Validate scheme from Excel]
    D2 -->|false| D4[Upload file only]
    D3 --> D5[IngestForm]
    D5 --> D6[SaveIngestedForm]
    D6 --> D7[Upload path triggers GMS2 + MQ]

    C4 --> Z[Done]
    C7 --> Z
    D4 --> Z
    D7 --> Z

Flow Details​

1) Standard SubForm flow (/elements/subform)​

Business context:

  • User uploads a standard subform for an application.
  • Depending on scheme rules, Elements either ingests data fields or treats it as upload-only.

Flow with code points:

  1. The subform is submitted either by the company in Portal or by the OIC in Elements. That submission reaches POST /elements/subform, mapped by [Route("subform")] in FormController.IngestSubForm (Edb4.AppService.Rest/Controllers/Elements/FormController.cs#L84), which hands control to SubFormService.IngestSubForm (Elements.ServiceFacade/Service/SubFormService.cs#L54).
  2. Elements then identifies which application this upload belongs to by loading the IVP record, and uses that IVP context to load the relevant scheme configuration through SchemeConfigUtility.Get in SubFormService (Elements.ServiceFacade/Service/SubFormService.cs#L54). From there, it decides whether this particular scheme should perform structured ingestion or should behave as upload-only (Elements.ServiceFacade/Service/SubFormService.cs#L105).
  3. Where ingestion is required, Elements reads the Scheme defined name from the uploaded Excel and checks that it matches the scheme on the application itself, using ivpEntity.Type as the application-side value, before moving into the shared ingestion path through SubFormService → IngestForm (Elements.ServiceFacade/Service/SubFormService.cs#L174).
  4. At the start of shared ingestion, FormHelperService.IngestForm narrows the available template definitions to the SubForm set, excluding PostDTA templates, so the upload is assessed against the correct type of form (Elements.ServiceFacade/Helpers/FormHelperService.cs#L145).
  5. Elements then reads FormVersion from the uploaded Excel in FormHelperService.Ingest and looks for the matching FormMaster row for that version (Elements.ServiceFacade/Helpers/FormHelperService.cs#L372). If no matching version exists, the process stops here and the upload is treated as an invalid template version.
  6. Once the matching FormMaster has been identified, Elements loads the related FormMasterFields rows for that FormMasterId (Elements.ServiceFacade/Helpers/FormHelperService.cs#L397). These rows are the field-by-field instructions that tell Elements what to read from the Excel file and how to interpret it.
  7. Elements then works through those FormMasterFields definitions in ValidateIngestion, reading each configured value from Excel, validating it, and preparing SQL inserts for FormSubmissionData (Elements.ServiceFacade/Helpers/FormHelperService.cs#L579). This is the point where the uploaded subform is converted into structured data.
  8. If no ingestion errors are found, Elements commits the prepared FormSubmissionData records through SubFormService → SaveIngestedForm and FormHelperService.SaveIngestedForm (Elements.ServiceFacade/Service/SubFormService.cs#L183, Elements.ServiceFacade/Helpers/FormHelperService.cs#L195).
  9. In parallel with that structured-data path, the original Excel file is also uploaded through the file service. Where the scheme is upload-only, this file-upload step still happens, but no FormSubmissionData is created.

2) GC template flow (/elements/postdtasubform)​

Business context:

  • User uploads GC template (PostDTA subform).
  • In current implementation, ingestion is enabled when application family is TaxCredit.
  • If ingestion succeeds, Elements persists structured values and then continues upload/handoff path.

Flow with code points:

  1. The GC template is uploaded by the OIC in Elements. That submission reaches POST /elements/postdtasubform, mapped by [Route("postdtasubform")] in FormController.IngestPostDtaSubform (Edb4.AppService.Rest/Controllers/Elements/FormController.cs#L126), which hands control to PostDTASubformService.Process (Elements.ServiceFacade/Service/PostDTASubformService.cs#L31).
  2. Elements then loads the IVP record to identify the application and uses that IVP context to load the relevant scheme configuration through SchemeConfigUtility.Get in PostDTASubformService (Elements.ServiceFacade/Service/PostDTASubformService.cs#L31). From there, it determines whether ingestion should run, which in the current implementation means TaxCredit applications (Elements.ServiceFacade/Service/PostDTASubformService.cs#L53).
  3. Where ingestion applies, Elements reads the Scheme defined name from the uploaded Excel and checks that it matches the scheme on the application itself, using ivp.Type as the application-side value, before continuing (Elements.ServiceFacade/Service/PostDTASubformService.cs#L56).
  4. Elements then enters the shared ingestion path through PostDTASubformService → IngestForm (Elements.ServiceFacade/Service/PostDTASubformService.cs#L82). At this point, FormHelperService.IngestForm narrows the available template definitions to the PostDTASubForm rows for that scheme (Elements.ServiceFacade/Helpers/FormHelperService.cs#L145).
  5. Elements reads FormVersion from the uploaded Excel in FormHelperService.Ingest and tries to match that version to the correct FormMaster row for the GC template (Elements.ServiceFacade/Helpers/FormHelperService.cs#L372). If no matching version exists in FormMaster, the process stops before any field parsing begins.
  6. Once the matching FormMaster is found, Elements loads the corresponding FormMasterFields rows for that FormMasterId (Elements.ServiceFacade/Helpers/FormHelperService.cs#L397). These rows define the field-by-field instructions for how the GC template should be read and validated.
  7. Elements then works through those FormMasterFields definitions in ValidateIngestion, reading each configured value from the Excel file, validating it, and preparing SQL inserts for FormSubmissionData (Elements.ServiceFacade/Helpers/FormHelperService.cs#L579). This is the point where the uploaded GC template is converted into structured data.
  8. If no ingestion errors are found, Elements commits the prepared FormSubmissionData rows through PostDTASubformService → SaveIngestedForm and FormHelperService.SaveIngestedForm (Elements.ServiceFacade/Service/PostDTASubformService.cs#L87, Elements.ServiceFacade/Helpers/FormHelperService.cs#L195).
  9. After the structured ingestion succeeds, the original GC template file is uploaded and stored inside Elements through the FileService path (Elements.ServiceFacade/Service/FileService.cs#L552). This is still part of the Elements-side upload flow.
  10. Only after the GC template has been stored in Elements does the PostDTA process move into downstream integration: FileService calls GMS2 process action (Elements.ServiceFacade/Service/FileService.cs#L565) and then publishes the TPGMS/GMS1 message queue event (Elements.ServiceFacade/Service/FileService.cs#L569).

Shared ingestion internals (used by both flows)​

  • PostDTA/SubForm FormMaster filtering happens in FormHelperService.IngestForm (Elements.ServiceFacade/Helpers/FormHelperService.cs#L145).
  • Uploaded Excel version is matched to the correct FormMaster row in FormHelperService.Ingest (Elements.ServiceFacade/Helpers/FormHelperService.cs#L372).
  • The related FormMasterFields rows are then loaded for that FormMasterId before field parsing continues (Elements.ServiceFacade/Helpers/FormHelperService.cs#L397).
  • Field-by-field extraction and validation happens in FormHelperService.ValidateIngestion (Elements.ServiceFacade/Helpers/FormHelperService.cs#L579).
  • Prepared FormSubmissionData SQL commands are committed in FormHelperService.SaveIngestedForm (Elements.ServiceFacade/Helpers/FormHelperService.cs#L195).

Data Model Involved​

FormMaster​

  • Purpose: Template metadata by type/version/scheme.
  • Used to decide whether uploaded FormVersion is valid for the current flow.
  • If no matching version exists, ingestion stops early.

FormMasterFields​

  • Purpose: Ingestion mapping rules for a specific FormMaster.
  • Defines where to read each value from Excel and how to validate it.
  • Key columns drive ingestion behavior: SheetName, Source, SourceType, DataType, IsMandatory, Key, and optional custom validation.

FormSubmissionData​

  • Purpose: Persisted structured key-value records extracted from template.
  • During ValidateIngestion, SQL insert commands are prepared for FormSubmissionData.
  • During SaveIngestedForm, those commands are executed and committed if no errors.

Ingestion Summary​

  1. Read FormVersion from Excel.
  2. Match uploaded version to FormMaster in scope.
  3. Load FormMasterFields for that FormMaster.
  4. Parse and validate each field from Excel.
  5. Build SQL commands for FormSubmissionData.
  6. Save via SaveIngestedForm if no errors.
  7. On errors, rollback and return validation/error output.

What gets saved​

  • Parsed values are prepared as FormSubmissionData rows.
  • The original Excel file is uploaded separately through the file service.
  • For PostDTA, downstream GMS2 / MQ handoff happens in the upload path.

Common failure points​

  • Scheme mismatch between Excel and IVP
  • Uploaded template version not present in FormMaster
  • Missing or incorrect FormMasterFields
  • Field validation failure during ValidateIngestion
  • Downstream failure after upload in PostDTA handoff path

Current logging notes​

  • Release branch logs are relatively minimal unless enhanced logging is deployed.
  • The most useful current failure points are usually:
    • PostDTASubformService.Process ...
    • ValidateIngestion error - ...
    • GMS2Service ProcessAction failed
  • With enhanced logging, you can see stage-level failures such as version match, ingestion, upload, and handoff.