Initial commit: Bale file agent demo

.freebuff/project-id

@@ -0,0 +1 @@
+88a10d2d-71e7-439a-9669-b247c744e387

n8n-Bale-file-fetch/HOW_IT_WORKS.md

@@ -0,0 +1,60 @@
+# Bale File Agent — How It Works
+
+## The idea
+
+The interviewer's brief was: type a natural-language request like *"grab the devops PDFs from this channel"* and have an agent go fetch them — the way a person would, but automated. This workflow implements that as a short pipeline in n8n, using an LLM only for the one step that actually needs language understanding (turning a sentence into a structured filter), and plain code/API calls for everything else.
+
+## The pipeline, step by step
+
+```
+Chat message
+   ↓
+LLM parses intent → {file_type, keyword}
+   ↓
+Fetch messages the Bale bot has seen
+   ↓
+Filter messages against the parsed intent
+   ↓
+Download each match from Bale
+   ↓
+Save to disk
+```
+
+### 1. Chat Trigger
+A live chat box inside the n8n editor. Whatever you type becomes the field `chatInput` on the item that flows to the next node. This stands in for "a person typing a request" — in a real deployment this could be a Slack bot, a web form, or Bale itself.
+
+### 2. Parse command (LLM)
+A single HTTP call to BazaarLink's OpenAI-compatible endpoint. The system prompt forces the model into a narrow, mechanical job: read the sentence, output nothing but `{"file_type": "...", "keyword": "..."}`. This is intentionally the *only* place an LLM is used — everything else is deterministic, which keeps the workflow fast, cheap, and easy to debug.
+
+### 3. Extract filters
+A small Code node that takes the LLM's raw text response, strips any accidental markdown fences, and parses it into real JSON so the rest of the workflow can use `.file_type` and `.keyword` as normal fields.
+
+### 4. Fetch Bale updates
+Calls Bale's Bot API (`getUpdates`) — a long-polling endpoint that returns every message the bot has seen since it was last called. Bale's bot API is essentially a clone of Telegram's, so this behaves the same way: it returns *new* activity, not full channel history.
+
+### 5. Filter matches
+Another Code node. Loops through the fetched messages, and for each one:
+- Checks whether it's a PDF document or a photo (matching `file_type`)
+- Checks whether the caption or filename contains the requested `keyword`
+- If both match, records the file's `file_id` and real `file_name` for the next steps
+
+This is where "understanding the request" turns into "acting on the request" — the filtering logic itself needs no AI, just the structured output the LLM already produced.
+
+### 6. Get file path → Download file
+Two calls to Bale's API mirroring Telegram's own two-step file download: `getFile` resolves a `file_id` into a temporary `file_path`, then a second request downloads the actual bytes from that path.
+
+### 7. Save to disk
+Writes the downloaded binary to a folder on the n8n host, using the real original filename (not Bale's internal file token) so the output is something a human would recognize.
+
+## Why it's built this way
+
+- **LLM only where it's needed.** Intent parsing is genuinely a language task; filtering, fetching, and downloading are not. Using an LLM for the whole thing would be slower, more expensive, and harder to debug than necessary.
+- **Deterministic filtering.** The match logic is plain code, so it's predictable and cheap to test — you can reason about exactly why a file did or didn't match.
+- **Small, inspectable steps.** Each node does one clear thing, so a failure (like a bad file path, or an expired long-poll) is easy to isolate — you can see exactly which stage produced bad data by clicking through node outputs in n8n's UI.
+
+## What a production version would add
+
+- A **persistent index** of channel content (log every incoming file to a small database or sheet as it arrives via webhook), so requests aren't limited to whatever `getUpdates` happens to still be holding.
+- **Retry logic** on the LLM call for transient network blips.
+- A proper **credential store** in n8n instead of hardcoded tokens/keys.
+- Possibly a **semantic match** (embeddings or a second LLM pass) instead of plain substring matching on captions, so "grab the devops stuff" also catches a file captioned "infrastructure automation" even without the literal word "devops."

n8n-Bale-file-fetch/SETUP.md

@@ -0,0 +1,67 @@
+# Bale File Agent — Setup Guide
+
+A demo n8n workflow that lets you type a natural-language request (e.g. "grab the devops pdfs") and have an agent fetch matching files from a Bale channel and save them to disk.
+
+## 1. Create the Bale bot and channel
+
+1. Open Bale and message **@Bot_Father** (same flow as Telegram's BotFather).
+2. Create a new bot, name it, and save the bot token it gives you — looks like `1234567:AAbbCCddEE...`.
+3. Create a private channel (this demo uses one called `lesson-source`).
+4. Add your new bot to the channel and **promote it to admin**. This step is mandatory — a bot that's only a regular member will never receive `channel_post` updates.
+5. Post a handful of test files into the channel with hashtag captions, e.g.:
+   - `#ansible` on one PDF
+   - `#linux` on another
+   - `#devops` on another
+   
+   Mixed captions let you prove the filtering actually filters, rather than matching everything by accident.
+
+## 2. Get a BazaarLink API key
+
+This workflow uses [BazaarLink](https://bazaarlink.ai) as an OpenAI-compatible LLM gateway, using the `auto:free` model (free, rate-limited — plenty for a demo where you're parsing short one-line commands).
+
+Grab an API key from your BazaarLink account and keep it handy for step 4.
+
+## 3. Import the workflow into n8n
+
+1. In n8n, go to **Workflows → Import from File**.
+2. Import `bale-file-agent-chat.json`.
+3. You'll see 7 nodes: a Chat Trigger, an LLM parser, a code node, three Bale API calls, and a disk-write node.
+
+## 4. Fill in your credentials
+
+Open these two nodes and edit them directly (no n8n credential objects needed for this demo):
+
+- **Parse command (LLM)** — replace the placeholder in the `Authorization` header with `Bearer <your BazaarLink key>`.
+- **Fetch Bale updates**, **Get file path**, **Download file** — replace the bot token in each URL with your own (`bot<YOUR_TOKEN>` in the path).
+
+## 5. Fix n8n's file-write allowlist
+
+n8n's Read/Write File node only allows writing to `/home/node/.n8n-files` by default — anywhere else gets rejected with "Access to the file is not allowed." To write to your own storage path, add this to your n8n service's environment in `docker-compose.yaml`:
+
+```yaml
+environment:
+  N8N_RESTRICT_FILE_ACCESS_TO: /mnt/storage
+```
+
+Then create the target folder and restart the container:
+
+```bash
+mkdir -p /mnt/storage/bale-downloads
+docker compose up -d --force-recreate n8n
+```
+
+## 6. Run it
+
+1. Open the workflow, click the chat icon on the Chat Trigger node (or the **"Open Chat"** button) to bring up an inline chat panel.
+2. Type something like `grab the devops pdfs`.
+3. Watch it run through: parse → fetch → filter → download → save.
+4. Confirm the file landed correctly:
+   ```bash
+   ls -la /mnt/storage/bale-downloads/
+   ```
+
+## Known limitations (worth knowing going in)
+
+- **`getUpdates` clears its queue on every call.** Once the workflow reads Bale's pending messages, they're gone from that endpoint until new ones arrive. If you run the workflow twice with no new channel activity in between, the second run will find zero files — not a bug, just how long-polling works.
+- **No historical scraping.** The bot only sees messages posted after it joined the channel. For files posted before the bot existed, you'd need a persistent index (log files into a database/sheet as they arrive) or a full client-level API integration — out of scope for this demo.
+- **Expression fields in n8n store a leading `=`.** If you ever copy/paste an expression into a field that's already in "expression mode," don't include your own leading `=` — n8n adds that internally, and a doubled-up `=` silently breaks the path.

n8n-Bale-file-fetch/json/Bale File Agent (demo).json

@@ -0,0 +1,234 @@
+{
+  "name": "Bale File Agent (demo)",
+  "nodes": [
+    {
+      "parameters": {
+        "options": {}
+      },
+      "id": "90a5fd0c-27ab-49be-ad12-87e53165df6a",
+      "name": "When chat message received",
+      "type": "@n8n/n8n-nodes-langchain.chatTrigger",
+      "typeVersion": 1.1,
+      "position": [
+        -80,
+        96
+      ],
+      "webhookId": "fdae82fa-cb2d-4d9a-83ba-4ae9bddd0a6e"
+    },
+    {
+      "parameters": {
+        "method": "POST",
+        "url": "https://api.bazaarlink.ai/v1/chat/completions",
+        "sendHeaders": true,
+        "headerParameters": {
+          "parameters": [
+            {
+              "name": "Authorization",
+              "value": "Bearer change-me-please"
+            },
+            {
+              "name": "Content-Type",
+              "value": "application/json"
+            }
+          ]
+        },
+        "sendBody": true,
+        "specifyBody": "json",
+        "jsonBody": "={{ JSON.stringify({ model: \"auto:free\", messages: [ { role: \"system\", content: \"You extract a file_type (pdf or image) and a keyword from a user's request. Reply with ONLY compact JSON like {\\\"file_type\\\":\\\"pdf\\\",\\\"keyword\\\":\\\"devops\\\"}. No prose, no markdown fences.\" }, { role: \"user\", content: $json.chatInput } ] }) }}",
+        "options": {}
+      },
+      "id": "64211cf6-e042-4e8b-a5ac-62d4e172c588",
+      "name": "Parse command (LLM)",
+      "type": "n8n-nodes-base.httpRequest",
+      "typeVersion": 4.2,
+      "position": [
+        368,
+        96
+      ]
+    },
+    {
+      "parameters": {
+        "jsCode": "let raw = $input.first().json.choices[0].message.content.trim();\n// strip markdown fences if the model adds them anyway\nraw = raw.replace(/^```json\\s*/i, '').replace(/^```\\s*/i, '').replace(/```\\s*$/i, '');\nconst filters = JSON.parse(raw);\nreturn [{ json: filters }];"
+      },
+      "id": "3aef64ed-d97c-4d23-b1d1-a06790ecf9f6",
+      "name": "Extract filters",
+      "type": "n8n-nodes-base.code",
+      "typeVersion": 2,
+      "position": [
+        592,
+        96
+      ]
+    },
+    {
+      "parameters": {
+        "url": "https://tapi.bale.ai/YOUR_BOT_ID:your-Bale-token/getUpdates",
+        "options": {}
+      },
+      "id": "639ac81a-f436-403c-8c4e-51ebb72b8547",
+      "name": "Fetch Bale updates",
+      "type": "n8n-nodes-base.httpRequest",
+      "typeVersion": 4.2,
+      "position": [
+        592,
+        336
+      ],
+      "notes": "getUpdates clears its queue on each call - if you run this twice in a row with no new channel activity, the second run comes back empty."
+    },
+    {
+      "parameters": {
+        "jsCode": "const filters = $('Extract filters').first().json;\nconst updates = $input.first().json.result || [];\n\nconst matches = [];\nfor (const u of updates) {\n  const msg = u.message || u.channel_post;\n  if (!msg) continue;\n\n  const isPdf = msg.document && (msg.document.mime_type || '').includes('pdf');\n  const isImage = !!msg.photo;\n  const caption = (msg.caption || msg.document?.file_name || '').toLowerCase();\n\n  const typeMatches =\n    (filters.file_type === 'pdf' && isPdf) ||\n    (filters.file_type === 'image' && isImage);\n  const keywordMatches = !filters.keyword || caption.includes(filters.keyword.toLowerCase());\n\n  if (typeMatches && keywordMatches) {\n    const fileId = isPdf ? msg.document.file_id : msg.photo[msg.photo.length - 1].file_id;\n    matches.push({ file_id: fileId, file_name: msg.document?.file_name || `${fileId}.bin`, caption, is_pdf: isPdf });\n  }\n}\nreturn matches.map(m => ({ json: m }));"
+      },
+      "id": "30f1df87-dce6-47c4-8110-51cfb5439feb",
+      "name": "Filter matches",
+      "type": "n8n-nodes-base.code",
+      "typeVersion": 2,
+      "position": [
+        800,
+        208
+      ]
+    },
+    {
+      "parameters": {
+        "url": "=https://tapi.bale.ai/YOUR_BOT_ID:chnage-me-please/getFile?file_id={{ $json.file_id }}",
+        "options": {}
+      },
+      "id": "3466c103-2079-4de3-b9a4-5fb1d034fa3a",
+      "name": "Get file path",
+      "type": "n8n-nodes-base.httpRequest",
+      "typeVersion": 4.2,
+      "position": [
+        1024,
+        208
+      ]
+    },
+    {
+      "parameters": {
+        "url": "=https://tapi.bale.ai/file/YOUR_BOT_ID:chnage-me-please/{{ $json.result.file_path }}",
+        "options": {
+          "response": {
+            "response": {
+              "responseFormat": "file"
+            }
+          }
+        }
+      },
+      "id": "19a4caeb-3ea7-4386-8c26-caab0061c2e2",
+      "name": "Download file",
+      "type": "n8n-nodes-base.httpRequest",
+      "typeVersion": 4.2,
+      "position": [
+        1248,
+        208
+      ]
+    },
+    {
+      "parameters": {
+        "operation": "write",
+        "fileName": "=/mnt/storage/bale-downloads/{{ $('Filter matches').item.json.file_name }}",
+        "options": {}
+      },
+      "id": "eab3cb34-f7ae-48c3-8b87-842a1e5fb176",
+      "name": "Save to disk",
+      "type": "n8n-nodes-base.readWriteFile",
+      "typeVersion": 1,
+      "position": [
+        1472,
+        208
+      ],
+      "notes": "Change the folder path to wherever your n8n instance can write. Swap for a Google Drive node if you'd rather store there."
+    }
+  ],
+  "pinData": {},
+  "connections": {
+    "When chat message received": {
+      "main": [
+        [
+          {
+            "node": "Parse command (LLM)",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    },
+    "Parse command (LLM)": {
+      "main": [
+        [
+          {
+            "node": "Extract filters",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    },
+    "Extract filters": {
+      "main": [
+        [
+          {
+            "node": "Fetch Bale updates",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    },
+    "Fetch Bale updates": {
+      "main": [
+        [
+          {
+            "node": "Filter matches",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    },
+    "Filter matches": {
+      "main": [
+        [
+          {
+            "node": "Get file path",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    },
+    "Get file path": {
+      "main": [
+        [
+          {
+            "node": "Download file",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    },
+    "Download file": {
+      "main": [
+        [
+          {
+            "node": "Save to disk",
+            "type": "main",
+            "index": 0
+          }
+        ]
+      ]
+    }
+  },
+  "active": false,
+  "settings": {
+    "executionOrder": "v1",
+    "binaryMode": "separate",
+    "availableInMCP": false
+  },
+  "versionId": "7098b660-f579-414b-bf83-6002d32cbff6",
+  "meta": {
+    "instanceId": "a8618b8efe9a0b8c68300d12dc12f140ace3ebe855b80198b7819ca3e0922fe9"
+  },
+  "nodeGroups": [],
+  "id": "tG4eEba8ODCbAcfX",
+  "tags": []
+}