How I Built My Own AI Family Office in ChatGPT (and How You Can Too)

Written By Daniel Mills

A step-by-step guide to turning ChatGPT into your personal financial advisory team.

When your finances straddle two countries, you need answers that no single adviser can give. U.S. tax code, Japanese residency rules, rental property portfolios, blue-form accounting—it’s a lot.
That’s why I’m building something new: a virtual family office inside ChatGPT.

Why I’m Doing This

I’ve spent years managing U.S. real estate, Japanese investments, and complex tax filings. I already have accountants and lawyers, but they work in silos. What I wanted was a central intelligence—a private workspace where every financial, legal, and tax perspective could meet and reason together.

That’s where ChatGPT’s Projects come in. A Project can hold documents, remember context, and act as a persistent “AI team.” I’m training mine to become the Mills-Kohyama Living Trust Family Office—an AI version of a cross-border wealth-management team.

The Vision: An Integrated Advisory Team

Inside this Project, I’ve defined virtual specialists:

  1. Chief Financial Strategist (CFP US & JP) – integrates all advice.

  2. Cross-Border Tax Director (CPA US & JP) – handles FEIE, FTC, PFIC, and treaty rules.

  3. Real Estate & Asset Manager (MBA – Real Estate Finance) – analyzes every property.

  4. Investment Architect – designs portfolios across NISA, iDeCo, IRAs, and taxable accounts.

  5. Legal & Estate Counsel (JD US & JP) – aligns trusts, inheritance, and entity structures.

  6. Insurance & Risk Analyst – models protection and umbrella coverage.

  7. Behavioral Finance Advisor – keeps the plan grounded in goals and values.

Every answer they produce follows a fixed format:
Executive Summary → Specialist Commentaries → Implementation Steps → Assumptions (with separate U.S./Japan sections and treaty citations).

Setting Up the Project

1. Create the Workspace

  • From ChatGPT’s home screen, click Projects → New Project.

  • Name it Mills-Kohyama Living Trust Family Office.

  • Paste your system prompt (the description of your AI team and output rules).

  • This Project becomes your permanent, private financial workspace.

2. Build the File Library

I divided everything into five folders:

/MillsKohyama_FamilyOffice
├── 01_ClientData
├── 02_ReferenceFrameworks
├── 03_ProfessionalResources
├── 04_Context
└── 05_Scenarios_Models

The Essential “Binding” Documents

Start with these eight core references—your AI’s legal and tax backbone:

FileWhy It MattersUS_Japan_Tax_Treaty.pdfDefines which country taxes what income.US_Japan_Treaty_Technical_Explanation.pdfClarifies each article’s meaning.IRS_Pub54_TaxGuide_Abroad.pdfCore guide for Americans overseas (FEIE, FTC, housing).IRS_Form2555_Instructions_2023.pdfImplementation details for FEIE.Japan_Income_Tax_Act_EN.pdfJapan’s actual tax law.NTA_Income_Tax_Guide_2024.pdfHow the NTA interprets and applies those laws.NTA_Blue_Return_Overview.pdfKey for business income and deductions.US_Japan_SocialSecurity_Totalization.pdfGoverns pension and coverage rules.

(Download from official IRS, Treasury, NTA, or SSA sites.)

Later, you can enrich the library with practitioner guides from KPMG, EY, or PwC and NTA treaty-application forms—but this “spine” is enough to start.

Upload and Index

  1. Upload the eight PDFs into your Project (in the order above).

  2. Tell ChatGPT:

    “Index these core references and create a one-page reference map summarizing each file, its key sections, and how it applies to cross-border financial planning for a U.S. citizen in Japan.”

  3. It will summarize and cite them—your first proof the system is working.

Activate the Virtual Family Office

Now issue this command:

“Activate the Mills-Kohyama Living Trust Family Office.
Summarize how you’ll use these references to handle:
(a) employment income, (b) U.S. rental income taxed in Japan,
(c) capital gains on U.S. assets, (d) FEIE vs FTC,
and (e) Social Security Totalization.
State default assumptions (FX, inflation, yields, tax bands).”

The AI will outline its understanding and assumptions—edit them once, and they’ll persist.

Test Runs

Try quick “smoke tests” to verify accuracy:

  • Treaty Articles Check –
    “Which treaty article governs dividends and capital gains, and what’s the summary rule?”

  • FEIE vs FTC Comparison –
    “Show two numeric examples comparing FEIE and FTC for an American living in Japan.”

If it answers correctly and cites sources, your family office is alive.

Expanding Over Time

Add your own spreadsheets later:

  • Investment_Portfolio.xlsx

  • RealEstate_Portfolio.xlsx

  • NetWorth_Snapshot.xlsx

  • Tax_Returns (US & JP)

  • Scenario_Retirement2030.xlsx

Each time you upload, prompt:

“Re-index today’s uploads and summarize new insights or discrepancies.”

That keeps your AI team current without starting over.

Why This Works

  • Persistent context: Projects remember your files and updates.

  • Private data: Everything stays in your workspace.

  • Interdisciplinary reasoning: AI blends tax, real estate, and investment logic.

  • Scalable: You can later add a separate Project for another trust or business.

Tips for Readers Who Want Their Own Version

  1. Pick a name for your “office” (e.g., Smith Family Office, Tanaka Advisory Lab).

  2. Create five folders as above.

  3. Start with the eight essential documents.

  4. Write a system prompt describing your virtual team and output structure.

  5. Test, refine, and expand—add your own spreadsheets once the legal framework works.

  6. Never upload sensitive IDs (use redacted or summarized data).

  7. Schedule quarterly updates just as a real adviser would.

Closing Thoughts

The financial world isn’t getting simpler. Cross-border investors, digital nomads, and expats often juggle two tax codes, two economies, and multiple currencies. Building an AI-powered family office inside ChatGPT doesn’t replace human professionals—but it unites them in one brain. That’s what I’m doing with the Mills-Kohyama Living Trust Family Office: creating a permanent, evolving command center for my wealth, taxes, and real estate across borders.
You can do it too—one project, eight documents, and a vision of financial independence made smarter by AI.

⚖️ Disclaimer

This article is for educational and informational purposes only and does not constitute financial, legal, or tax advice.
Readers should consult licensed professionals (CPA, CFP, tax attorney, etc.) before making financial or legal decisions.
The tools and methods described here are examples of how AI can support personal organization and learning, not a substitute for personalized advice.

Daniel Mills
Next

What We Can Learn from TIGER 21’s Asset Allocation