decant

Pour your Microsoft 365 into plain markdown, sediment left behind.

A Bun CLI that mirrors Microsoft 365 content into a local kb/ folder as markdown, so agents running on this machine can read company knowledge without calling Microsoft Graph themselves.

To decant is to pour the wine off the sediment. That is the job: Outlook's link wrappers were a third of one vault by weight, a notification's icons are named by machine ids nothing can read, and a picture's words belong under the picture rather than in a file of their own. What lands in kb/ is what a reader, or an agent, can actually use.

It syncs SharePoint document libraries, your Outlook mailbox (one markdown file per conversation, with its attachments and the SharePoint files it links to), your Microsoft To Do lists (one markdown file per task), the channels of the Teams you belong to (one markdown file per post, with its replies beneath it), the people directory (one markdown file per colleague, plus the org chart), your Outlook calendar (one markdown file per event, with what it carried beside it), the OneNote notebooks of your sites and your own (one markdown file per page), the lists of your SharePoint sites (one markdown table per list) and your Planner plans (one markdown file per task under its bucket, plus the board as one table).

Graph access goes through ask-marcel-office-cli used as a library, which owns authentication, paging and document conversion. See docs/marcel-library-map.md.

Requirements

• Bun 1.2 or newer
• A Microsoft 365 account. If you are not signed in, decant opens a browser where you sign in
• Python 3 with rapidocr installed, to read text out of images and scanned PDFs (optional; without it the files are still copied). It fetches the models it needs on first use, one per language it reads in

Install

The package on npm is ask-marcel-decant. The command that it installs is decant.

bun add --global ask-marcel-decant
decant --help

Or run it without installing anything:

bunx ask-marcel-decant

To change the code, clone the repo and run bun install. In a clone, bun run sync replaces decant in the commands below. bun test runs the suite.

Run

decant

Lists what you can read, grouped by what it is: the SharePoint sites, the Loop workspaces, your OneDrive, the inboxes of the Microsoft 365 groups you belong to, your Microsoft To Do lists, the Teams you are a member of, the OneNote notebooks of those sites and your own, the lists of those sites, and the Planner plans of the groups you belong to. Each is marked with when it last synced and how much it holds. Pick a number and one or more libraries (or, for a Team, one or more channels), press m for your mailbox, c for your calendar or p for the people directory, and it syncs.

Your Loop workspaces are on that list too, shown as Loop - <workspace>. A workspace keeps its pages in a container no site listing returns, so they are found through the .pod manifest each one carries; from there a workspace is synced exactly like a site, and update refreshes it with the rest. A workspace nobody has opened may be missing from the index for a while: sync it by container id with --site-id when that happens.

Several sites can be taken at once: 1,3 for those two, or all for every site on the list. Taking more than one site takes every library in each without asking, since the point of choosing them together is not being asked once per site. Each site is summarised as it finishes.

To refresh everything already synced, without being asked anything:

decant update

That form is safe to schedule: it never opens a browser, and a lapsed sign-in ends the run with a clear message instead of waiting for input. Any other source that fails is named as it fails, listed in kb/_sync-report.md under the ones to try again, and passed over, so the sources after it are still refreshed; the run then exits with a failure.

How far back

The first run that has something to sync asks how far back to reach: a day such as 2025-01-01, or all. The answer is kept in kb/.decant.json, and every later run reaches back to the same day without asking, update included. From that day on, decant syncs mail, group inboxes, Teams posts, the calendar, library files and OneNote pages, each by its own date: a message by the day it arrived, a group thread by its last post, a post or a page by its last change, a file by its last edit, an event by the day it starts (a recurring series is kept whatever day it began, since it may be running still). To Do lists, Planner plans, SharePoint lists and the people directory always come whole: they say how things stand now, and a board missing its old open tasks would mislead. Days are counted in UTC, the calendar's in the mailbox's zone.

To move the day, run once with --since, which keeps the new day for every run after it:

decant update --since 2024-06-01

Moving the day earlier fetches what the narrower reach left out, on each source's next run: a source that keeps a cursor reads it out first, so nothing deleted since the last run goes unnoticed, then reads the whole source once and writes only what it had left out. Moving the day later deletes nothing: what is already in kb/ stays. update and a run that names a source by flag never ask; with no day kept yet, they reach everything. Clearing kb/ clears the day with it, and the next run asks again. A day that cannot be read, mistyped by hand in kb/.decant.json, stops the run rather than syncing everything; --since sets it again.

Options

| Option | Meaning | |---|---| | --site-id <id> | Sync this site without showing the picker | | --site-url <url> | Sync the site at this address, for sites the search index does not list | | --drive-id <id> | Sync only this library; repeat for several | | --mailbox | Sync your Outlook mailbox without showing the picker | | --people | Sync the people directory, everyone in your Teams, without the picker | | --calendar | Sync your Outlook calendar without showing the picker | | --group-id <id> | Sync one group inbox without the picker, by its id or its address | | --todo-list <name> | Sync one To Do list without the picker, by its name or its id | | --team <name> | Sync every channel of one Team without the picker, by its name or its id | | --notebook <name> | Sync one OneNote notebook without the picker, by its name or its id | | --lists <site> | Sync the lists of one SharePoint site without the picker, by its name, its id or its address | | --plan <title> | Sync one Planner plan without the picker, by its title among the plans listed or by its id | | --since <day\|all> | Reach back to this day (2026-01-31), or through everything, and keep it for every later run (see How far back) | | --dry-run | Report what would be done and write nothing | | --max-size-mb <n> | Skip files larger than this (default 50) | | --concurrency <n> | How many items to convert at once (default 4); 1 is strictly sequential | | --no-ocr | Do not read text out of images or scanned PDFs | | --refresh | List the sites afresh instead of drawing the picker from the ones last seen | | --ocr-lang <code> | Force one language for images and scanned PDFs (default auto, chosen per image) | | --timezone <zone> | IANA zone the mailbox counts its days in, e.g. Asia/Shanghai (default: this machine's) |

What lands in kb/

kb/
  .decant.json                      how far back the runs reach: a day, or all
  _sync-report.md                   what the last run left behind, every source in one file
  _archive/SharePoint sites/<Site>/<Library>/...
                                    files whose source was deleted, renamed, or changed away
  SharePoint sites/<Site>/
    .sync-state.json                what has been synced, and where the next run resumes
    _sync-report.md                 what did not make it in, and why (only when there is something)
    _members.md                     who can open the site
    <Library>/
      2026-05-12/                   the day each document last changed at the source
        <SharePoint folders mirrored>/
          Roadmap.pptx.md           slide text
          Roadmap.pptx.pdf          the rendered deck
          Contrat.pdf               the original
          Contrat.pdf.md            its text layer
          Contrat.docx.md           the text, plus a section listing the pictures inside
          Contrat.docx.media/       those pictures, kept as files
  Loop workspaces/<Workspace>/      the same shape, for a Loop workspace
  OneDrive/<Owner>/                 the same shape, for a personal OneDrive
  Group inboxes/<Group>/            a group's conversations (see the mailbox layout below)
  Mailbox/                          your own Outlook mailbox
  Calendar/
    .sync-state.json                the delta cursor, and every event filed
    2026-09-12/                     the day each event starts
      Offsite planning.md           the event: when, where, who, how to join, what it is about
      Offsite planning.attachments/ what the invitation carried, as markdown
  People/
    .sync-state.json                who is filed, and a fingerprint of what each page was made of
    _org-chart.md                   everyone under their manager, linked to their pages
    Jane Doe.md                     one page per colleague
  To Do/<List>/
    .sync-state.json                which tasks are filed, and what each one last looked like
    2026-09-08/                     the day each task last changed
      Book the venue.md             the task, its notes, its steps and what it links to
  OneNote/<Notebook>/
    .sync-state.json                every page filed, and what it last looked like
    <Section group>/<Section>/      the notebook's own structure
      Kick-off.md                   one page
  SharePoint lists/<Site>/
    .sync-state.json                every list filed, and a fingerprint of what its table was made of
    Projects.md                     one table per list
  Planner/<Plan>/
    .sync-state.json                every task filed, and the etags its page was made from
    _board.md                       the whole plan as one table, linking each task
    <Bucket>/
      Book the venue.md             one task, under the lane it sits in
  Teams/<Team>/
    .sync-state.json                one delta cursor per channel, and every post filed
    <Channel>/
      2026-09-10/                   the day each post last changed, a reply counting as a change
        Venue for the offsite.md    the post and its replies, as one thread

Sources are shelved under the same headings the picker offers them under, so a source is found again under the heading it was chosen from. A Microsoft 365 group is a site and an inbox wearing one title, and the two halves land on different shelves: its documents under SharePoint sites/, its conversations under Group inboxes/. Reports and summaries still spell out (group inbox) after the name, where a bare name would leave a reader guessing which half they are reading.

Every document is filed under the day it last changed at the source, not the day it was synced, with the folders it had in SharePoint underneath: the day is what you usually want to scan by, and the folders are what keep two same-named documents apart. A document whose source reports no date at all goes under undated/.

Because the day comes from the document itself, editing one at the source files it afresh under its new day. The copy under the old day is not left behind as a duplicate: it is moved into _archive/, where anything the source no longer has already goes.

Two different SharePoint sites can share a display name (most often an unedited template title). The picker shows each one's address alongside the name so they can be told apart before choosing, and if a sync would otherwise land in a folder another site's .sync-state.json already claims, it is written under a disambiguated one instead (<Site>-<hash>/) rather than merging into it.

Every generated markdown file opens with where it came from. A source that is a real SharePoint or OneDrive link carries ?web=1, so clicking it opens the browser viewer directly instead of prompting to launch a desktop app; a mail conversation or a linked-drive-item source, neither a real URL, is left as its plain label.

---
source: https://tenant.sharepoint.com/sites/X/Roadmap.pptx?web=1
site: Espace Contoso
library: Documents
path: Projets/Roadmap.pptx
last_modified: "2026-05-12T09:31:00Z"
modified_by: Jane Doe
synced_at: "2026-07-23T14:00:00Z"
pdf: ./Roadmap.pptx.pdf
---

How each kind of file is handled

| Source | What you get | |---|---| | docx, doc, csv, odt/odp, msg, txt, md, html, json, sarif, xml, yaml, log | one markdown file | | xlsx, xls, xlsm, ods | the workbook, plus markdown holding its cell text | | loop, fluid, whiteboard | one markdown file, rendered by Graph since the page holds no text of its own | | pptx | markdown for the text, plus a PDF of the slides (markdown alone when the source will not render it) | | ppt, rtf | a PDF, plus markdown read back from it | | pdf | the original, plus markdown holding its text layer | | ics | one markdown record of the meeting: what it is called, when, where, who was asked | | url | one markdown file linking where the shortcut goes, when that is a web address | | eml | a folder: the message as markdown, and every file it carried taken out of the base64 it travelled in | | zip | a manifest listing every member with the text read out of it, one markdown file per document inside, and the archive itself | | jpg, png, gif, webp, bmp, tiff, heic | the image, plus markdown holding the text read out of it | | svg | the file, plus a markdown note pointing at it | | anything else | left in SharePoint and named in _sync-report.md |

A workbook is kept as it came as well as read. The conversion yields cell text and nothing else, so a formula, a second sheet, a number format and a chart survive nowhere but the file itself; the markdown names it under original:. Every other document kind is markdown alone, since for text the markdown is the content.

A saved email is unpacked the way an archive is. The library has no parser for raw MIME and hands the file back whole, which means every attachment sits in the middle of the text as base64, so the message is read here instead: the header block, the text in whatever encoding a 7-bit transport forced on it, and each file it carried written as a file and converted like any other. A file of a kind nothing can read keeps the file and loses only the text.

A meeting invitation is read rather than kept. What survives is the summary, the time exactly as the invitation states it (no timezone is converted, since there is no zone database here to do it with), the location, the organiser, everyone asked, the recurrence rule, and whether the invitation is a cancellation. What goes is the block of daylight-saving rules, the vendor properties, and the description, which repeats the mail the invitation rode in on.

The language an image is read in is chosen per image, not per run. Nothing about a file says which language it holds, so the run finds out by reading it twice. The first reading uses the widest dictionary RapidOCR ships, which holds ideographs, both Japanese syllabaries and accented Latin at once, so it can say what the image is written in. That reading is then thrown away and the image is read again by the model built for what it found: Chinese, Japanese, or Latin, the last covering the accented characters Dutch, French and German need and keeping the spaces between words that a Chinese model runs together. The model that recognises everything is the best at nothing, which is why it only ever decides and never answers.

Each markdown companion records the model that read it, as ocr: rapidocr (ch), (japan) or (latin). An image holding no text at all is read only once, since there is no script to get right. What each image read to is kept under kb/_meta/ocr, addressed by the image's own bytes, so it is read once and never again: a picture renamed, moved, or arriving a second time as somebody else's attachment answers from the same entry, and a re-sync costs no interpreter starts at all. The language is part of the address, so forcing one never reads back what a previous run decided for itself. Only successful readings are kept, since a failure says something about the machine rather than about the image.

Pass --ocr-lang with a RapidOCR language (ch, en, japan, korean, cyrillic, ...) to force one for the whole run and skip the deciding pass. A language RapidOCR does not have is refused straight away, with the accepted ones listed, rather than failing once per image.

Korean is the one script the deciding pass cannot recognise: no model that reads Korean also reads anything else, so the first reading returns Chinese-looking nonsense for a Korean page instead of returning nothing. Sync a Korean source with --ocr-lang korean and it reads correctly.

A picture inside a document does not survive the conversion to markdown: it becomes a bare [image], usually with no alt text, so an architecture diagram contributes one word that says nothing. Word and Excel files are therefore also asked for the pictures they embed, which land in a <name>.media/ folder beside the markdown and are listed under an ## Images heading at its end. Each is read for its text the same way a photo in a library is, so the labels inside a diagram become searchable; with --no-ocr the pictures are still kept, just without the reading. The placeholders are unnumbered, so the section sits at the end rather than pretending to know where in the prose each picture stood.

This holds on both sides of the sync: a Word file attached to a mail loses its diagrams exactly the way one sitting in a library does, so it is asked the same question, and the pictures land in the same <name>.media/ folder beside the same markdown. They are not named in the head of the thread, which lists what each message carried: the document's own markdown links them, and a Word file with a dozen diagrams would otherwise put a dozen lines at the top of a conversation.

Only those two kinds are asked. A PDF is kept whole beside its markdown and a deck is rendered to one, so their pictures are already on disk in openable form: taking them out again would duplicate what is there, and it costs the most exactly where it buys the least, since one 14 MB manual answers with 172 images and tens of minutes of reading. Legacy doc, xls and ppt are refused by the source, and a plain-text kind has nothing embedded to ask about.

A document carries more than the text it shows. Every conversion, in SharePoint and on mail attachments alike, also asks for the side channel the rendered body hides: comments and threaded replies, tracked insertions and deletions, hidden text, external hyperlinks, and a flag for embedded macros. It arrives as a metadata section under the document's text. Office formats (and their macro-enabled and template variants) and OpenDocument carry one; on every other kind of file there is nothing to add and the request changes nothing.

Who can open a site

Each SharePoint site gets one _members.md beside its libraries. The data comes from list-sharepoint-site-members. The page gives:

• The Microsoft 365 group of the site, and the owners and members of that group.
• The SharePoint groups on the document library, with their roles.
• The sharing links on the document library, with their roles.

# Who can open Espace Contoso

## Owners

- [Jane Doe](<../../People/Jane Doe.md>) ([email protected])

## Members

- Jane Doe ([email protected])
- Sam Lee, guest

## SharePoint groups

- Contoso Team Visitors: read

Graph does not list the persons in a SharePoint group. Thus the group members are the minimum set of persons who can open the site. Graph shows a caller who is not a site owner only the grants that apply to that caller. A file with unique permissions is not in this page.

If the People sync wrote a page for a person, the name links to that page. The links come from the last People sync. A person without a page at that time gets a link on a later run.

Loop workspaces and OneDrives do not get this page, because they have no Microsoft 365 group. If Graph does not give the data, the run keeps the old page and continues. If the answer does not change, the run does not write the page again. If the site gives an empty answer, the run moves the old page to _archive/.

What did not make it in

Anything left behind is named in _sync-report.md beside the source it came from, newest run first: files of a kind this tool does not read, files above the size cap, files locked with a password, the stubs Office leaves beside an open document, files that could not be read, and files moved aside because the source no longer has them. A file that could not be read is written down and converted again on the next run, up to three tries in all; after that it is named under a heading of its own saying it will not be tried again, and it keeps that line in every report until the file changes at the source. A password is the one refusal that never becomes readable by trying again, so such a file is left out the way an unsupported type is, rather than queued afresh on every run. A ~$Budget.xlsx sitting beside Budget.xlsx is the same case decided earlier: Office writes it while the workbook is open and it holds the editor's name rather than the document, so it is named for what it is instead of reaching the converter, failing as a malformed workbook, and coming back on every run. A run that converted everything writes nothing there, so a nightly sync does not bury the runs that did leave something behind.

kb/_sync-report.md, one level up, holds the same thing for the run as a whole, so twenty sources do not mean twenty files to open. It is rewritten on every run rather than appended to, and always covers every source: the ones the run touched get their counts and their lists, and every other source already in kb/ is named with the date it last ran, so a source that was not rechecked is never mistaken for one with nothing wrong. A group inbox is named as one, Northwind Projects 2026 (group inbox), because the site of the same name is a separate source with its own section, and two identical headings would leave a reader guessing which held the documents. A run that left something behind ends with a line on the terminal saying so and naming the file. Its history is the per-source files; this one is the current view. A dry run writes neither.

What a mailbox sync writes

kb/
  Mailbox/
    .sync-state.json                a cursor per folder, and what every conversation produced
    _sync-report.md                 what did not make it in, and why
    _inline/                        pictures pasted into messages, stored once for the mailbox
                                    and named by content, since one signature logo rides on
                                    every message its sender ever wrote
    threads/2026-05-12-ca0df2c95a-contrat-contoso/    the day it began, its id, what it is about
      contrat-contoso.md                             the whole conversation, oldest message first
      _attachments/Contrat.docx                      a file a message carried, under its own name
      _attachments/Contrat.docx.md                   its card: who sent it, when, and its text
      _linked/Rapport.docx                           a SharePoint file a message pointed at
      _linked/Rapport.docx.md                        its card: the address, the version, its text
    _meta/threads.jsonl                              one line per thread, for querying
    _meta/attachments.jsonl                          one line per stored file, by content address
    _meta/links.jsonl                                one line per document pointed at

What a To Do sync writes

One markdown file per task, filed under the day the task last changed, the same way a document is:

---
source: To Do - Tasks
list: Tasks
status: completed
importance: high
due: "2026-09-12"
completed: "2026-09-08"
created: "2026-08-01T09:00:00Z"
last_modified: "2026-09-08T16:20:11Z"
categories:
  - Work
synced_at: "2026-09-09T14:00:00Z"
---

# Book the venue

Ask for the September rate.

## Steps

- [x] Shortlist three
- [ ] Call them

## Links

- [Contract draft](https://tenant.sharepoint.com/sites/X/Contract.docx)

Completed tasks land alongside the open ones, each marked with its status and the day it was finished. A knowledge base is for remembering what was decided, and most of that is in work already done; a sync that kept only the open tasks would delete a task from kb/ at the moment it stopped being a plan and became a fact.

An ordinary task says nothing about its importance, so the word only appears where it means something. A task with no notes, no steps and nothing linked carries none of those headings rather than empty ones.

Editing a task files it afresh under its new day, and the copy under the old day moves into _archive/ rather than being left behind as a duplicate. A task deleted in To Do goes to _archive/ too, and is named in the list's _sync-report.md.

Each list is read whole every run. To Do does have a delta endpoint, but it answers with a task's own fields and leaves out its steps and its linked resources, which are navigation properties and have to be expanded; one paged read that expands both costs less than a delta plus a fetch per changed task, and it is also what makes a deleted task visible, by its absence, on every run rather than on the single run a delta reports its removal.

What a calendar sync writes

One file per event as the calendar holds it: a single meeting, a recurring series as one file carrying its rule in words, an exception to a series as its own file. Expanded occurrences would be fifty-two identical files for a weekly meeting.

---
source: https://outlook.office365.com/owa/?itemid=...
subject: Offsite planning
start: "2026-09-12 15:00"
end: "2026-09-12 16:30"
organizer: Valerie
attendees:
  - Jane Doe (accepted)
  - Dana Farrow (optional, no answer)
location: Rotterdam, Room 3
online_meeting: https://teams.microsoft.com/l/meetup-join/...
recurrence: every week on Monday, from 2026-01-05 until 2026-12-31
my_response: accepted
synced_at: "2026-09-12T14:00:00Z"
---

# Offsite planning

Agenda: ...

## Attachments

- [Venues.xlsx](<Offsite planning.attachments/Venues.xlsx.md>)

Times are given in the zone the calendar counts its days in, the same --timezone the mailbox uses; an all-day event is given as days. The body is the invitation's text with the markup gone, so the Teams boilerplate reads as a few lines rather than a page of HTML.

The calendar delta reports what changed and no more, so each event it names is fetched in full. A reply, a reschedule or a new attendee moves the event's lastModifiedDateTime, the delta reports it, and it is written again, refiled under its new day if the start moved, the old copy going to _archive/. A deleted event goes to _archive/ too and is named in the report; a cancelled one stays, marked cancelled: true, since a cancelled meeting is still a fact.

What an invitation carries lands as markdown only, beside the event: the library converts an event attachment to text and nothing fetches its bytes, so there is no PDF and no original the way a mail attachment has. A picture pasted into an invitation is left out and named in the report as such. The first run walks the whole calendar and writes the events starting on or after the kept day, and every recurring series whatever day it began (see How far back). The cursor moves only once every event the delta reported has landed.

What a people sync writes

One page per colleague, and one org chart:

---
source: Microsoft 365 directory
name: Jane Doe
title: Head of Operations
department: Operations
manager: Dana Farrow
email: [email protected]
phones:
  - "+31 6 1234 5678"
office: Rotterdam
teams:
  - Leadership
  - NORTHWIND EMPLOYEES
synced_at: "2026-09-11T14:00:00Z"
---

# Jane Doe

Head of Operations, Operations. Reports to [Dana Farrow](<Dana Farrow.md>).

## Reports

- [Ann Lee](<Ann Lee.md>)

There is no directory listing in the library (/users is not exposed), so the people are found through the Teams you belong to: the roster of every one of them, joined by user id, is the directory. With an all-employees Team that is everyone; without one it is everyone you work with. Each person then costs one call, get-user with the manager expanded into it. Guests and disabled accounts are read and left out, and count as nobody's report.

Graph offers no delta on people the library exposes, so every run reads every profile again. What makes a run an update rather than a rewrite is a fingerprint per person in the state file, over the profile, the Teams and the reports: an unchanged person costs a call and no write, a changed title or manager rewrites that one page, a newcomer is written, and someone whose account is disabled or who has left every Team is moved to _archive/People/ and named in the report. The org chart is drawn again whenever any page was.

What a OneNote sync writes

One file per page, under the notebook's own structure rather than under a day, since a page is a place and not a moment:

---
source: https://tenant.sharepoint.com/sites/lead/SiteAssets/Notebook#Kick-off
notebook: Northwind Leadership Notebook
section: Meetings
group: "2026"
title: Kick-off
created: "2026-08-01T09:00:00Z"
last_modified: "2026-09-01T10:00:00Z"
synced_at: "2026-09-12T14:00:00Z"
---

# Kick-off

Agenda ...

The notebooks offered are your own and those of every site the picker knows, one call per site made side by side; a site that refuses OneNote reads (a tenant past OneNote's item limit answers every read with a refusal) costs its own notebooks and not the listing. Two notebooks can carry one name, since every site's default notebook is named after the site, and the second to be synced takes an id-suffixed folder the way two sites sharing a name do.

Pages have no delta, so every run lists every section and every page again and rewrites a page only when its lastModifiedDateTime moved; a page renamed or moved between sections is written afresh and the old copy goes to _archive/, and a page gone from the notebook goes there too and is named in the report. A section whose pages cannot be listed keeps its pages on disk as they are.

A page of your own notebooks renders through the library, pictures embedded. A page of a site notebook is answered as HTML only, with no renderer in the library for it, so its text is read out here the way a calendar event's body is, and a picture in it reads as a link to a Graph resource that needs a sign-in to open. Section groups inside a site notebook are out of reach on the site route, so the pages inside one are not offered. Both are gaps in the library rather than here.

What a SharePoint lists sync writes

One file per list, the whole list as one markdown table, since a tracker is read as a table and not one row at a time:

---
source: https://tenant.sharepoint.com/sites/contoso/Lists/Projects
site: Espace Contoso
list: Projects
description: Every project we run
rows: 2
last_modified: "2026-09-11T10:00:00Z"
synced_at: "2026-09-12T14:00:00Z"
---

# Projects

Every project we run

| Title | Status | Owners | Sponsor |
|---|---|---|---|
| Falcon | Open | Jane Doe | #12 |
| Eagle | Done |  |  |

Every site in the picker is offered for its lists, under its own heading and apart from the same site's libraries, which are the drive sync's business: a document library is a list in Graph's eyes and is passed over here, as are the hidden lists SharePoint keeps for itself. What is taken is the kind a person makes and fills in (a custom list, a task or issue tracker, a calendar of events, contacts, links, announcements, a survey).

The columns are the ones a person sees on the list: the title, and whatever is neither hidden nor read-only, so SharePoint's own bookkeeping (a colour tag, a compliance id, who edited last) stays out. A date reads as its day, a flag as yes or no, a person or a lookup by name, a link as a link. A column holding one person or one lookup is answered by Graph as an id alone, shown as #12, since the name is not in the row; a column holding several is answered by name.

Lists have no delta, so every run reads every list again and rewrites a table only when a row's lastModifiedDateTime moved, a row came or went, or the columns changed. A list renamed is written afresh and the old table goes to _archive/, a list gone from the site goes there too and is named in the report, and a list whose rows cannot be read keeps its table on disk as it was.

What a Planner sync writes

One file per task under the bucket it sits in, since a plan is a board and the lanes are how a person reads it, plus _board.md, the whole plan as one table:

---
source: https://planner.cloud.microsoft/webui/plan/<plan>/view/board/task/<task>
plan: Offsite 2026
bucket: To do
progress: in progress
priority: urgent
labels:
  - Waiting on client
assigned_to:
  - Jane Doe
  - Sam Lee
start: "2026-09-01"
due: "2026-09-30"
created: "2026-08-20T09:12:00Z"
created_by: Jane Doe
synced_at: "2026-09-12T14:00:00Z"
---

# Book the venue

Three rooms, one with a projector.

## Checklist

- [x] Ask for a quote
- [ ] Sign the contract

## References

- [venues.xlsx](<https://contoso.sharepoint.com/sites/x/Shared Documents/venues.xlsx>)

Progress is Planner's three words, priority its four bands (medium left unsaid, being the default), dates are days, and assignees are named through one directory read per person per run. Labels are named from the plan's details, read once a plan per run, on the page and in the board's last column; a label nobody named is a colour in Planner and is left out rather than shown as category3, and renaming one rewrites the pages that carry it. The details come through get-planner-plan-details; when that read is refused, the plan still syncs, without its labels, and the log says so (labels.unnamed).

The plans offered are the ones Graph lists for you and those of every group you belong to, one call per group through list-group-planner-plans. On the tenant this was built against your own listing answers nothing and the groups hold the plans, so the group route is the one that fills the picker. Both commands arrived in ask-marcel-office-cli 2.8.0 (docs/request-group-planner-plans.md). --plan <id> reaches one plan without the picker, the id being what the address bar shows.

Tasks have no delta, so every run lists the buckets and the tasks again and reads every task's details, the description having an etag of its own that the card does not carry; a task is rewritten when either etag moved or its page belongs under another lane now, put aside when it left the plan, and the board is drawn again whenever any page was. A plan with hundreds of tasks costs hundreds of calls a run.

What a Teams sync writes

A Team is chosen the way a site is, and then asked which of its channels to take the way a site is asked about its libraries. Each channel gets a folder, and each root post in it one markdown file:

---
source: https://teams.microsoft.com/l/message/19%3Aabc%40thread.tacv2/1757491200000
team: Northwind Leadership
channel: General
subject: Venue for the offsite
author: Jane Doe
created: "2026-09-10T08:00:00Z"
last_modified: "2026-09-10T09:30:00Z"
synced_at: "2026-09-11T14:00:00Z"
---

### 2026-09-10 08:00 · Jane Doe

**Venue for the offsite**

Three options, see below.

> **Dana Farrow · 2026-09-10 09:30**
> The second one.

The thread under the front matter is rendered by ask-marcel-office-cli itself, the way a mail message is: the Teams HTML converted, @mentions flattened to their names, attachment placeholders resolved to links, each reply quoted beneath the post oldest first, with _(edited)_, _message deleted_ and _reactions: 👍 2_ markers where they apply. A picture pasted into a post is a placeholder; the bytes live in Graph's hosted content and are not fetched.

A post is filed under the day it last changed, and a reply counts as a change: Graph moves the root post's lastModifiedDateTime when a reply lands, so a thread that gains a reply is written again whole under its new day and the copy under the old day moves to _archive/. A post deleted in Teams goes to _archive/ too, and is named in the team's _sync-report.md. A post is named by its subject when it has one, which most do not, and otherwise by its first line.

Each channel keeps a delta cursor of its own, so a re-run asks Graph only for what changed. The cursor moves only once every post the delta reported has landed: a post that would not render is named in the report, and the channel re-reads its delta next run to try it again, skipping by lastModifiedDateTime everything that already landed.

Only the channels of Teams you are a member of are offered, and only their standard, private and shared channels as Graph lists them. Teams chats (one-to-one and group chats) are a different thing and are not synced: the library can only reach them through Microsoft-internal endpoints whose token expires daily, which would fail every scheduled update.

What a group inbox sync writes

The same, in a folder of its own:

kb/
  Group inboxes/
    Leadership Team/
      .sync-state.json              what has been filed, and the newest post it reaches
      threads/2026-07-20-abc1234567-bi-monthly-leadership-meeting/
        bi-monthly-leadership-meeting.md             every post, oldest first

A Microsoft 365 group is three things wearing one name: a SharePoint site holding files, an inbox holding conversations, and a Teams team. The picker offers the first two separately, and the shelf is what keeps them apart on disk: both carry the group's name, one under SharePoint sites/, the other under Group inboxes/.

Only the groups you belong to can be read. Access is decided by membership rather than by any permission this tool can ask for, so a group the directory lists but you have not joined refuses every read, and it is not offered.

A post attachment gets everything a mail attachment gets: a deck is rendered to a PDF beside its text, a diagram inside an attached Word or Excel file is extracted and read, and a SharePoint link in a post body is pulled into _linked/. The three commands this needs arrived in ask-marcel-office-cli 2.6.0; before it, each answered with an honest empty.

Group threads are not tracked by a delta the way a mailbox folder is, because Graph offers none. Each run lists a group's threads newest first and stops at the newest post it already holds, so a thread that has been replied to comes back and an untouched one does not.

A thread that could not be written is remembered and asked for again on the next run, three tries in all, whatever the dates say. It has to be remembered rather than found again: a thread that failed records nothing, so a newer thread that did land carries that stopping point past it, and no reading of the dates alone would ever reach it. The same holds for a thread that lands while one of the files it carried does not. After the third try the thread is named once under the heading that says it will not be tried again, by its topic rather than its id.

A thread folder is self-contained: everything a conversation carried or pointed at sits inside it, beside a card that says how it arrived. The one exception is a picture pasted into a message, which lives in _inline/ for the whole mailbox, because a signature logo rides on every message its sender ever wrote and a copy per thread would write the same hundred kilobytes two hundred times. Everything else is written into every thread that received it. That is the trade this layout makes on purpose: a report sent to ten threads is stored ten times, and no folder has to reach into another to be read.

The card and the file share a name. Contrat.docx.md carries what belongs to the arrival, who sent it, when, under which message, and the text read out of the file, so a reader opening the folder finds one document per thing rather than a card and an extract saying the same. When the file itself was kept, a workbook or a PDF, the card names it under original: and links it. Two files of one name in a single thread, a form resent after correction being the usual way, are both kept and the second is numbered. A card is still written for a file too large to fetch or of a kind nothing reads, since it is the only record that the thread depended on something the knowledge base does not hold; a file nothing can read that is also named by a machine id, the icons a sharing notification is built from, is dropped without a card, a line or a report entry, there being no fact left to record.

One file per conversation, not per message, in a folder named once and never renamed: the day of its first message, a ten-character id, and a readable slug of its subject. A reply appends to the document and leaves the folder alone, so a path into kb/ keeps working. Recency lives in last_message in the front matter, not in the path.

The id is a hash of the thread's root Message-ID, taken from the References header of its oldest message, rather than Graph's conversationId. conversationId is scoped to one mailbox, so a shared mailbox and a personal one see different values for the same message, and Graph reassigns it when an external party replies from outside Exchange.

That last case is why it matters: Graph opens a second conversation for the same exchange, and both resolve to the same root, so both are written into one document. Every conversation a thread was assembled from is listed as conversation_id in the front matter, and nothing is keyed on it.

Which thread a conversation belongs to is settled once, before anything is written, and never revisited. A message arriving late, older than anything already held, would otherwise answer with a different root and rename a folder already on disk. A conversation whose headers cannot be read is left unresolved and retried on the next sweep rather than filed under a guess.

The day is counted in the zone given by --timezone, defaulting to this machine's. Every timestamp Graph returns is UTC, so a message received at 16:40Z is the next day in a UTC+8 tenant, and the folder name is written once.

Every folder is swept except Junk, Deleted Items, Drafts and Outbox, which means the mail you sent is included: Sent Items is a folder like any other, and a conversation is assembled from every folder its messages landed in. Quoted reply chains are stripped, since the message being quoted is already its own section.

A SharePoint file a mail merely points at is pulled too, into the thread's own _linked/, flat, beside its card. The card records the address at the source, when the document last changed and who changed it, and carries its text, so a thread can be traced to a version of a document and not only to its name. A linked deck arrives as a PDF as well as its text, and a linked file above the size cap is left where it is with a card saying so. What somebody shares is as often a folder as a file: a folder gets a card that says a folder was shared, and no entry in the report, since nothing went wrong.

Each message's own section names the files that message carried, linking each card, so a reader following a conversation sees what arrived with which reply rather than one list at the head of the file. A picture pasted into a message is shown where it stood, or after the text when the conversion lost the placeholder that said where, and never listed as a file somebody sent. What was read out of it is quoted under it, behind one line saying the words were read by a machine and may be wrong: OCR read pe off a logo saying something longer, and without that line the two letters read as text somebody wrote. A picture under 10 KB is not read at all. Measured across one mailbox, everything OCR found real words in was 64 KB or more and everything under ten was a logo, so the threshold costs no content and saves a twenty-second pass per logo.

A saved email attached to a message is unpacked into a folder of its parts, and its card shows its pictures where the message showed them, read the same way, and lists every file it carried under Carried by this message:, linking the reading where one was made and the file where it was not.

Every link a message carries is written as the address the sender sent. Outlook rewraps each one through its own scanner with a per-recipient tracking blob after it, and measured across one mailbox that wrapper was a third of everything written and left no link readable. Unwrapping means a click from the vault reaches the destination without Outlook's check at click time; this is an archive of what was sent, and what was sent is the destination.

The head of each thread names where the file itself lives under path:, so a thread pasted into a context window can still be resolved and traced back, and lists every participant with the address you would reply to. Message headings keep names alone, since one heading already runs to eleven recipients.

_sync-report.md holds only what a reader should look into. A run in which every file arrived writes the counts and the line "Nothing was left behind." A file left out says which kind it was, a .mp4 this tool does not read, or that its name had no extension so nothing could tell.

A conversation that could not be written is remembered and rendered again on the next run, the same three tries a SharePoint document gets, and named by its subject under the same two headings: the one that promises another run, then the one that says there will not be another. One that failed before decant kept subjects goes by its thread id, since that is all the state file holds for it. A reply arriving on a thread that had run out of tries starts it over.

A thread that lands while one of the files it carried does not is remembered the same way, and the whole thread is written again on the next run to fetch that one file. It costs the conversions of the files that already worked, which land on the same paths they landed on before, and there is no narrower way back: the folder cursors moved on when the messages were swept. After the third try the file is named once as given up and the thread is left alone; the card in the thread's own _attachments folder goes on saying what happened to it.

A first mailbox run is slow: Outlook hands back a folder a hundred messages at a time, and only because every page asks for it again, since the size is honoured only on the request that carries it. A mailbox with tens of thousands of messages takes hundreds of round trips, and every conversation written for the first time costs one more, to find the thread it belongs to. Later runs are cheap, reading only what changed. The kept day narrows what gets written, not what gets swept.

While it works, a counter on the terminal shows how far it has got, so a long run is never silent. A header row carries the count, and every item still being read from the source gets a row of its own beneath it, naming the step it is on:

SW Project (Fabrikam instance) / 文档  4/25 (8 running)
  Projets/PT Findings.xlsx · reading picture 3/6
  General/04_IT_Security_overview/Overview.docx · rendering the slides

A mailbox counts twice before it writes anything: its folders as they are read, each with the messages read so far, then the conversations it has not seen before, by subject, while it finds the thread each belongs to. Once it writes, each conversation's row names the attachment or linked document it is reading, since a dozen screenshots take minutes to read. A first run spends most of its time on the first two:

Mailbox, reading folders 3/14
  Inbox · 1200 messages read

The block rewrites itself where it stands as items come and go, so one slow file does not look like the run has stopped while its --concurrency siblings finish around it. Set --concurrency higher than the terminal has rows and the overflow collapses into a single …and N more row, so the block never outgrows the screen. It draws only when stderr is a terminal, so piping the output to a file or running headless leaves no counter behind.

Re-running

Each run stores the cursor Graph gives it, so a second run reads only what changed: an unchanged library converts nothing. State is written after every single file, so stopping a run mid-way (Ctrl-C) loses at most the file in flight, and the next run picks up from the same place. A file renamed in SharePoint is moved on disk rather than converted again; a file deleted there has its markdown moved into kb/_archive/, as does the older copy of a file that was edited and so now belongs under a later day.

kb/ is generated content and is gitignored.

Configuration

Read once at startup, in src/composition/config.ts:

| Variable | Meaning | Default | |---|---|---| | KB_ROOT | Folder the knowledge base is written to | kb | | KB_LOG_LEVEL | Winston level; logs go to stderr, output to stdout | error |

Development

This repo follows the atelier standard: Clean Architecture, TDD with hand-written fakes, Result<T, E> at every IO boundary. See CLAUDE.md.

bun test                       # test suite, in a new random order on each run
bun run lint                   # fast rules, zero warnings allowed
bun run lint:strict            # adds the type-aware rules (pre-commit gate)
bun run typecheck              # tsc --noEmit
bun run coverage               # per-tier gate: 100% domain and use-cases, 80% elsewhere
bun run mutate                 # Stryker over everything, break threshold 90
bun run mutate:changed         # Stryker over what changed since origin/main, each file held to 90
bash scripts/verify-commits.sh # every commit since origin/main typechecked and tested on its own

bun test runs the tests in a new random order each time, because bunfig.toml sets randomize = true. A run that fails prints --seed=<n>. bun test --seed=<n> runs the tests again in the same order.

Stryker breaks on the total of the files it mutates, so one weak file passes while the others carry it. mutate:changed and mutate:staged follow the run with scripts/check-mutation-files.ts, which reads the report the run just wrote and fails on any single file under the break. The full sweep on main keeps the total alone.

The pre-push hook runs scripts/verify-commits.sh over what is being pushed: each commit is checked out alone in a detached worktree and has to typecheck and pass the suite there, since the pre-commit hook only ever sees the working tree. A commit touching only prose (*.md, .claude/, docs/) is skipped.

Lint holds every function to a complexity of 10 and every layer to the imports the dependency rule allows: the domain imports no other layer, the use cases nothing beyond the domain and their own layer, the presenter only the domain, infra neither the presenter nor the composition root, and no production file reaches the test fakes, which themselves never wrap an adapter. It refuses a class, an inline type import, a curried arrow other than a createX factory, a try in a use case, node:fs in src/ outside infra, the fakes and the tests, a @ts- comment and another tool's ignore marker, and toEqual([]) in a test, which passes on [undefined]; a comment disabling a rule does nothing but get reported itself. eslint.config.test.ts lints a violation of each gate through the real config, so a gate that stops firing turns the suite red. Stryker runs bun test src, which leaves those fixtures out: they exercise the config, not a line of src/.

Layout

src/
  domain/        pure logic: paths, front matter, conversion planning, delta diffing, state
  use-cases/     orchestration against ports; ports/ holds the port types
  infra/         adapters: the Graph library, Bun filesystem, RapidOCR, Winston, stdin, stdout
  presenter/     pure renderers, handed to the use cases through a port
  composition/   option parsing, configuration, dependency wiring
  test-helpers/  hand-written fakes and builders
  main.ts        entry point, the one top-level catch