# TAF ERP This repository contains the ERP system used across multiple counsels. ## Environment Setup Codex development environments should provision Docker by running the automated setup script before starting the stack: ```bash sudo ./setup_docker.sh ``` The script installs Docker, verifies `docker compose` is available and invokes `./start_docker.sh --detached` so the `.env` file is generated and containers boot in the background. After the initial setup you can rerun `./start_docker.sh` whenever you need to rebuild or restart the services. ## Environment Variables Certain modules require environment variables. For Microsoft 365 integration, set the following variables before running the application: - `M365_TENANT_ID` – Azure tenant identifier. - `M365_CLIENT_ID` – application client ID. - `M365_CLIENT_SECRET` – application client secret. - `M365_GRAPH_URL` – Microsoft Graph API base URL (optional, defaults to `https://graph.microsoft.com/v1.0`). Copy `config/m365.php.dist` to `config/m365.php` and ensure these environment variables are present in your deployment environment. Database access requires setting the `DB_DRIVER` environment variable: - `DB_DRIVER` – database driver (`pgsql`, `mariadb`, `oci`, `mariadb_mongo`). Defaults to `pgsql` if not specified. The test helper script uses this value to install the correct packages. See `docs/DatabaseConfig.md` for connection options and Mongo settings. - `DISABLE_AUDIT_LOGS` – deprecated. Set `logs_disabled` to `true` in `config/audit.php` to disable audit log writes. When working across database engines, import `TAFDB.sql` into your server and run `DB_DRIVER= ./test_setup.sh` to configure drivers. See `docs/DatabaseConfig.md` for details. ## Database Migrations After loading the base schema (`TAFDB.pgsql` for PostgreSQL or `TAFDB.sql` for MySQL) apply the SQL files under `migrations/` in numeric order. These scripts create new tables such as `table_column_settings` and alter existing ones. Once the schema is migrated, run: - `20250901_allow_null_tenant_tables.sql` allows `tenant_id` to be nullable in `tenant_tables` so definitions can be shared across tenants. - `20251029_create_workflow_triggers.sql` adds the `workflow_triggers` table used by multi-form workflows. - `20260110_seed_resources.sql` populates the `resources` table with existing modules, forms and tables. - `20260133_add_resource_type_to_permissions.sql` adds the `resource_type_id` column to `permissions` and backfills existing rows. - `20260602_seed_legacy_modules.sql` registers legacy modules from `FrontEnd/Dashboard.php` in `ui_modals` and `resources` so they can be assigned per tenant. ```bash psql -f migrations/20260602_seed_legacy_modules.sql php cli/pgsql_schema_sync.php ``` This command syncs the schema and seeds `sql/defaults.sql` so the initial super admin account and other default settings exist. Pass `--no-seed` to skip loading the defaults. Legacy modules from `FrontEnd/Dashboard.php` now appear in `ui_modals` and the `resources` table, so permissions can assign them to specific tenants. When upgrading from an earlier release, run any new SQL files under `migrations/` and then execute `php cli/pgsql_schema_sync.php` again. This ensures new resources are seeded and sequences remain in sync. ## LocalSyncManager & Search Platform The 2026 sync refactor introduced a policy-driven `LocalSyncManager` singleton, an event bus, and a shared search helper. Key expectations for new work: - Always obtain the manager via `services/syncservice` (`getManager()`, `requestData`, `registerPolicy`) instead of instantiating `core/localsyncmanager` directly. See `docs/frontend/LocalSyncManagerModuleStatus.md` for migration progress across modules. - Policies define cadence, retention, retry and optional search metadata. Store overrides through the policy registry (`services/syncpolicies`). - Listen for `sync:*`, `queue:*`, and `search:*` events on the shared event bus to keep UI elements (loaders, toasts, diagnostics) in sync. - Use `services/searchHelper` for combined local and remote lookups. The helper respects policy-configured search columns and emits progress events. - Queue writes while offline; the orchestrator replays them automatically when connectivity returns. Avoid bespoke retry loops in modules. Progressive hydration is rolling out through ModuleLoader/DataLoader updates. When adding new data sources, prefer `requestDataSource` over bespoke `fetch`/`$.ajax` calls so sync policies and hydration remain consistent. ## Module Platform & Progressive Loading - Module definitions now live in the `modules` table. Use ModuleBuilder to author configs and ModuleLoader to resolve them—new features should not add entries to `ui_modals`. - Declare module data sources and let ModuleLoader kick off progressive hydration (skeletons, incremental renders, pagination) instead of manual fetch chains. - Keep events scoped to the module controller. Use schema `data-action` hooks or module emitters/listeners rather than `window.*` callbacks. - Extend ModalBuilder layouts using the shared component registry (grids, computed state, analytics widgets) before reaching for bespoke markup. - Call `requestDataSource`/`requestData` for IndexedDB-backed reads so policies and auto-refresh remain consistent across modules. - When touching legacy modules, consult `docs/frontend/LocalSyncManagerModuleStatus.md` to confirm whether the singleton manager and ModuleLoader patterns are already in place. - Apply the tenant→org terminology migration whenever you update module data, store names, analytics, or API payloads to keep the stack consistent. ## UI, Forms & Accessibility - Share form element type constants between FormBuilder, FormUtils, and ModalBuilder; avoid hard-coded DB strings in modules. - Use DaisyUI/Tailwind components with their accessibility defaults (focus states, ARIA attributes) and ensure new layouts support keyboard navigation. - Adopt the lightweight file upload widget and document preview components for simple attachments; route large workflows through the Document Management Service. - Surface offline/diagnostic messaging via the event bus-driven helpers instead of custom banners. - Apply the action guard helper when binding buttons or forms to prevent double submits and race conditions. ## Security, Auth & API Usage - Use JWT-based auth for REST, sync, and websocket routes; renew tokens on login and keep them in secure storage (IndexedDB/session). - Update websocket connections to include JWT validation, jittered retries, and fallback polling before surfacing errors to users. - Rely on the shared `services/api` helper when composing endpoints so base-URL overrides work consistently during multi-org routing. - Wrap REST calls with `services/apiClient` so transport/validation errors are normalised and surfaced via Toast messages instead of uncaught exceptions. - Ensure DMS endpoints, queued uploads, and file widgets enforce the same JWT middleware and return standardized error payloads. The Outlook calendar integration and Teams meeting links used by the Interview module depend on these credentials. Provide your tenant ID, client ID and client secret obtained from Azure AD. The file `config/m365.php` is loaded by `OutlookCalendarService` and `TeamsService` to connect to Microsoft Graph. ## Tenants and Branches TAF supports multiple counsels in a single deployment. Each organisation is a **tenant** with one or more **branches**. Most tables now include `tenant_id` and `branch_id` columns so data stays isolated per counsel. See `docs/MultiTenantMigration.md` for an overview of the schema changes and how to upgrade existing databases. ## Tenant Identification Requests are mapped to the correct tenant using the request host or the first URL segment. When DNS entries exist you may access `tenant.example.com`, but if subdomains are not configured you can prefix the request path, e.g. `example.com/tenant`. The middleware strips that segment and continues processing. Tenants may also define their own domains via the `tenants.custom_domain` column. `TenantRegistry` checks this value when resolving hosts so a tenant can use `mytown.gov` instead of a subdomain. Set `BASE_DOMAIN` to your deployment's primary domain and `ADMIN_PORTAL_HOST` to the URL of the administration portal. These values are used when generating links and during tenant resolution. Super administrators may also supply optional branding values when creating a tenant. The `product_name` appears on the login page while `dashboard_name` labels the main dashboard. A custom `logo_url` replaces the default logo image for both locations. If these settings are omitted the stock branding defined in `FrontEnd/index.html` and `FrontEnd/JsLibs2/modules/Dashboard.js` is used. Example SQL to create a tenant that uses its own domain and includes optional branding values: ```sql INSERT INTO tenants (name, custom_domain) VALUES ('New Counsel', 'newcounsel.gov') RETURNING id INTO :tenant_id; INSERT INTO settings (name, value, tenant_id) VALUES ('product_name', 'Acme ERP', :tenant_id), ('dashboard_name', 'Acme Dashboard', :tenant_id), ('logo_url', 'https://example.com/logo.png', :tenant_id); ``` Or via the API: ```bash curl -X POST https://admin.example.com/api/tenants \ -H 'Content-Type: application/json' \ -d '{ "name": "New Counsel", "custom_domain": "newcounsel.gov", "product_name": "Acme ERP", "dashboard_name": "Acme Dashboard", "logo_url": "https://example.com/logo.png" }' ``` The login page reads `product_name` and logo information, while the dashboard uses `dashboard_name`. Supplying these settings ensures a branded experience per tenant. ## API Authentication All API requests must include an `Authorization` header containing a JWT token. The token payload includes the user ID along with `tenant_id` and `branch_id` so the backend can scope queries correctly. Obtain this token by calling `/api/auth/login` or the OTP login endpoint. ## Deduction Settings Payroll deductions can be configured by administrators. Visit `views/hr/deduction_settings.php` and use the interface to add or edit rules. The page communicates with the `/api/deduction_settings` endpoints. Each rule specifies a salary range, fixed amount, percentage of gross pay and optional employer contribution. Rules may optionally include a start and end date and an "applies to" designation of Resident, Non-Resident or Everyone. These amounts are automatically applied by `PayrollProcessor` during pay runs. ### Country vs Company Rules Deduction settings can be defined globally for a country or specifically for a single company. Country level rules require the `country_deductions.manage` permission and apply automatically to all companies in that country. Company specific rules still use the `deduction_settings.manage` permission. Both sets of rules are visible in the same interface – country rules appear in a read–only table for most users, while company rules can be added or edited if the user has the proper permission. Country holiday dates are managed with the `country_holidays.manage` permission. Super admins can add or edit holidays through the **Country Holidays** module on `FrontEnd/Dashboard.php`. ### Dashboard Card Permissions Dashboard cards no longer include `data-acl` attributes. Visibility is controlled by rows in the `dashboard_card_permissions` table and applied server side. The Dashboard receives only the cards the current user is allowed to view. Use the Card Permissions Manager modal to manage which groups or users can view a card. Tenant entries grant access only to administrators of that tenant. A record with `group_id=0` represents an "all groups" rule that applies to every user unless a specific group or user entry overrides it. User and group permissions take precedence over tenant and global group rules. Legacy module visibility is now enforced by the `AuthorizationEngine` on the server. Custom modules defined in `ui_modals` load dynamically once the engine grants access, so the dashboard markup no longer contains every possible card. The built-in Super Admin account (`user_id` 1) bypasses these module checks and always sees every module. ### Permissions Manager The dashboard now includes a **Permissions** card. Clicking it loads the Permissions Manager module where administrators assign access policies to users or groups. See [docs/AccessPolicies.md](docs/AccessPolicies.md) for full instructions. ## CLI Jobs Several command line scripts are intended to run on a schedule: 1. **Payroll Scheduler** – `api/jobs/schedule_payroll.php` should run daily via cron. It checks `pay_cycles.next_run_date` and queues payroll runs when due. 2. **Email Ingest** – `cli/email_ingest.php` processes configured mailboxes and stores attachments in the Documents module. Run it every 15 minutes or as needed. 3. **Performance Demo Data** – run `php cli/performance_demo.php` to seed example goals, reviews, feedback notes and check‑ins. `deploy.sh` will execute this script when called with the `--demo-performance` flag. 4. **OHS Dashboard Demo Data** – run `php cli/ohs_demo.php` to insert sample incidents and workflows used on the safety dashboard. 5. **JSA Incomplete** – `cli/jsa_incomplete.php` emails supervisors when JSA sign‑offs are overdue. Configure the grace period via the `JSA_GRACE_DAYS` environment variable. 6. **Permit Expiry Alerts** – `cli/permit_expiry.php` warns applicants about upcoming or lapsed permits. Set `PERMIT_EXPIRY_DAYS` to adjust the warning window. During automated tests you can export a short value such as `PERMIT_EXPIRY_DAYS=1` so sample data triggers immediately. 7. **Scheduled Reports** – `jobs/scheduled_reports.php` generates saved reports according to entries in the `scheduled_reports` table. 8. **Vehicle Service Alerts** – `jobs/vehicle_service_alerts.php` emits `service.due` events for upcoming maintenance. Set `SERVICE_ALERT_DAYS` to adjust the lead time. Example cron line: ```cron 0 6 * * * /usr/bin/php /path/to/jobs/vehicle_service_alerts.php >> /var/log/service_alerts.log 2>&1 ``` 9. **Database Schema Sync** – `cli/pgsql_schema_sync.php` creates or updates tables based on `TAFDB.pgsql` and applies `sql/defaults.sql` so default data exists. Run it after configuring your database connection. Use the `--no-seed` option if you only want the schema changes applied. Ensure `config/mailboxes.php` contains your IMAP connection details before running the email ingest job. ### Outgoing Email Workflow notifications use PHPMailer for SMTP delivery. Configure your SMTP credentials using the service connection settings API. POST a payload like ```json { "type": "smtp", "credentials": { "host": "smtp.example.com", "port": 587, "username": "user@example.com", "password": "secret", "from": "no-reply@example.com", "from_name": "TAF ERP", "encryption": "tls" } } ``` `WorkflowEngine` resolves the `smtp` connection via `ServiceConnectionSettingsController` and `ConnectionFactory` when sending email steps. ## Service Connections Service connections store credentials for external services such as SMTP or SharePoint. Each connection can be defined globally, per tenant or per user, and the settings API resolves them in that order. Manage records through `/api/service_connections` or the administration UI. The dashboard includes a **Service Connections** modal that lists existing records and provides an **Add Connection** button. See [docs/ServiceConnections.md](docs/ServiceConnections.md) for details on inheritance and adding drivers. ## Time Clock API The `timeclocks` resource tracks employee clock ins and outs. To fetch records within a specific period, call: ``` GET /api/timeclocks/by-range?start=YYYY-MM-DD&end=YYYY-MM-DD ``` Parameters: - `start` – inclusive lower bound timestamp (YYYY-MM-DD or full datetime). - `end` – inclusive upper bound timestamp. The response is filtered based on the caller's permissions and returns an array of matching time clock rows. ## OHS-to-T&A Integration Workflows triggered from the OHS module can update the Attendance system. Set `ENABLE_OHS_ATTENDANCE=1` in your environment and import a workflow like `docs/ohs_absence_workflow.json`. Injury events will then create leave requests and adjust attendance records using the `create_leave_request` and `update_attendance` steps. ## Recruitment Module See docs/RecruitmentModule.md for a summary of candidate intake and hiring features. ## Performance Management API Endpoints under the performance module follow a common pattern. When a record ID does not exist the API responds with an empty array rather than a 404 error. Affected resources: - `/api/employees` - `/api/performance_reviews` - `/api/performance_goals` - `/api/feedback_notes` - `/api/check_ins` See `docs/PerformanceManagementAPI.md` for examples. ## Notification Log All emails, SMS alerts and calendar reminders are recorded in the `notification_log` table. When a calendar event is created the system queues a reminder entry using `EventNotificationService`. Run `api/jobs/send_event_reminders.php` daily via cron to deliver upcoming reminders. Example cron line: ``` 0 8 * * * /usr/bin/php /path/to/api/jobs/send_event_reminders.php ``` ## Notification Queue Changes to records now insert a row into the `notification_queue` table. The `NotificationQueueService` is responsible for writing these entries. Generic CRUD controllers enqueue notifications automatically and workflows may push their own messages using a `queue_notification` step. See [docs/NotificationQueue.md](docs/NotificationQueue.md) for field details. The processing worker requires the phpRedis extension; installation steps are listed in [docs/NotificationQueue.md#dependencies](docs/NotificationQueue.md#dependencies). The related test suites are summarised in [docs/RealTimeTests.md](docs/RealTimeTests.md). ## PDF Generation Some modules export data to PDF documents. The Training Certificates module uses [Dompdf](https://github.com/dompdf/dompdf) to convert HTML templates. Install the library via Composer and ensure the autoloader is available: ```bash composer require dompdf/dompdf ``` Place the generated `vendor` directory in the project root so `vendor/autoload.php` can be loaded by the controllers. ### Payslip Signing Payslips can optionally include an image watermark and signature. Provide the path to your signature image in `config/payroll.php` under `payslip_signature_path` or update the setting via the **Payroll Management** module. Uploading a new image will store it under `/uploads/signatures` and update the setting automatically. The PDF signing functionality depends on the `setasign/fpdi` library. Install it alongside Dompdf: ```bash composer require setasign/fpdi ``` ## Kanban Board Tasks can be organised on a drag‑and‑drop board divided into columns. Each board belongs to a tenant and can be restricted to a department. Workflows may create tasks automatically using the `create_task` step type. See [docs/KanbanModule.md](docs/KanbanModule.md) for table details and API examples. ## Modal Builder Dynamic modals can be defined through a JSON schema and rendered with `ModalBuilder`. This helper supports forms, tables and raw HTML sections. See [docs/frontend/ModalBuilder.md](docs/frontend/ModalBuilder.md) for the schema format. Forms and modal layouts are stored in the `ui_modals` table and loaded at runtime. `ModalBuilder` renders them client-side and synchronises data immediately through the `/api/sync` bulk endpoints. `clients` and `cases` are implemented entirely with `GenericController` and depend on these bulk endpoints for CRUD operations. Both modules sync through `/api/sync` and use modal forms resolved from `ui_modals`. ### Metadata Responses Generic CRUD endpoints now include a `metadata` block whenever the model defines one. DataLoader reads this information automatically and processes `booleanFields`, `relations` and `calculatedColumns`. See [docs/ModelMetadata.md](docs/ModelMetadata.md) for the structure. ```json { "data": [/* records */], "metadata": { /* field info */ } } ``` ## Column Visibility DataLoader tables include a right‑click menu on the header row for hiding or showing columns. Hidden columns can be enforced at the Super Admin or tenant level and further customised by each user. Preferences are stored in the `table_column_settings` table and managed through `/api/table_column_settings`. See [docs/frontend/ColumnVisibility.md](docs/frontend/ColumnVisibility.md) for full details. ## Definition Caching Workflow and form definitions are cached in memory via `CacheService`. The service stores values in an in-memory array and uses APCu when available. Controllers automatically invalidate the cache whenever a definition is created, updated or deleted. Restart PHP or call `CacheService::clear()` if you need to manually flush cached entries. Cached items expire after 5 minutes by default. Pass a TTL to `CacheService::set` or `remember` to override. ## Form Builder and Workflow Engine The **Form Builder** lets administrators design forms without editing code. Definitions are saved in the `forms` table with each form's fields stored in the `form_elements` table (or serialized in the `elements` JSON column). The builder UI is implemented in `FrontEnd/JsLibs2/modules/FormBuilder`. Metadata models for these records live under `models/`. Submission data is stored in `form_submissions` with field values in `form_submission_values`. `WorkflowEngine` executes automated steps in response to system events. Workflow records are stored in `workflows` with their actions in `workflow_steps`. The engine loads definitions from the database, evaluates conditions and performs actions like sending emails or updating records. `SettingsService` resolves options using a department → branch → tenant → super hierarchy. Modules such as theming rely on this service to provide the correct setting value for the current user. For a detailed description of this inheritance model see `docs/HierarchicalConfiguration.md`. ## Detail View Builder Custom detail pages can be arranged with the drag-and-drop **Detail View Builder**. Layouts are stored in the `detail_views` table and loaded at runtime. See [docs/DetailViewBuilder.md](docs/DetailViewBuilder.md) for the palette, canvas and inspector layout along with save/load examples. The `Counsel-System` directory is deprecated and exists only for reference. Leave its contents untouched. ## Graph Builder The **Graph Builder** module lets administrators build different chart types (scatter, line, bar, stacked bar and pie). Select a table, choose the chart type and drag columns onto the X, Y, Radius and Label drop zones. Configurations can be saved through `/api/graphs` and loaded by ID. A **Download PDF** button exports the current chart. See [docs/GraphBuilder.md](docs/GraphBuilder.md) for usage details. The documentation now covers grouping and aggregation options. ## Fishbone Diagram The **Fishbone Diagram Manager** allows administrators to create and edit cause-and-effect diagrams stored in the `fishbone_diagrams` table. Diagrams are edited on an SVG canvas where categories and causes can be dragged to reorder them. Double-click any label to rename it and use the `+` or `✕` icons to add or remove items. The edit dialog also lets you pick a color for each branch, which controls the stroke and text color when rendered. A sample diagram is inserted by `sql/defaults.sql` for reference. See [docs/FishboneDiagram.md](docs/FishboneDiagram.md) for more information. ## Five Whys The Five Whys Manager documents iterative questioning when analysing an incident. Selecting **Add Whys** opens form **83** where you choose the incident and record up to five responses. Entries sync locally via `LocalSyncManager` so `SyncController` manages the `/api/sync` CRUD endpoints. See [docs/FiveWhys.md](docs/FiveWhys.md) for details. After modifying any builder, workflow or settings functionality run: ```bash phpunit -c api/phpunit.xml npm test ``` ## Testing Run `./test_setup.sh` after any change to provision the local testing environment. The script installs PHP and Node tooling, including the Jest package via `npm install`. It uses `DB_DRIVER` to determine which database service to start and will initialise credentials unless `SKIP_DB_SETUP` is set. `test_setup.sh` defaults to `DB_USER=TAF`, `DB_PASS=TAF123*` and `DB_NAME=TAFDB`. Override these variables if your local database uses different credentials. PostgreSQL is started automatically when `DB_DRIVER=pgsql`; ensure the service is available on port `5432` or set `DB_HOST`/`DB_PORT` accordingly. Example invocations: ```bash DB_DRIVER=pgsql ./test_setup.sh # default DB_DRIVER=mariadb ./test_setup.sh DB_DRIVER=oci ./test_setup.sh DB_DRIVER=mongo ./test_setup.sh ``` Execute the tests manually with: ```bash phpunit -c api/phpunit.xml npm test ``` The Jest environment runs under jsdom and is configured in `tests/js/setup.js` which exposes jQuery globally so test suites relying on `$` work without additional imports. Use `npm test -- ` to execute a single suite. - Server-side exceptions are captured via `Helpers\ErrorReporter`, logging structured metadata while returning generic error payloads to the UI. - Form element metadata now lives in `core/elementTypes.js` (mirrored in `sql/defaults.sql`); reuse those constants in FormBuilder/ModalBuilder/FormUtils instead of hard-coded strings when adding new types. - Expand Jest coverage for progressive hydration, search helper usage, and offline/online transitions by leveraging the IndexedDB/fetch/websocket mocks under `tests/js`. - Add PHPUnit cases for JWT middleware, modules API endpoints, and terminology migrations whenever those areas change. Only the modules you modify must have their tests run unless a full or integration suite is specifically requested. Tests will continue to evolve as modules are refactored. When working on a module, run the tests for that module. Full or integration suites are only required when specifically requested. As modules are refactored, update or extend their tests accordingly. For details on calendar endpoints see `docs/CalendarEventsAPI.md`. Usage of the Module Builder filter options is described in `docs/ModuleBuilderTasks.md`. The Safety Incidents workflow is covered in `docs/SafetyIncidents.md`.