VMware

Teaching the VCF JSON Generator About Non-vSAN Storage

Posted by

The VCF.JSONGenerator module is a great companion tool for the VCF 9.1 Planning & Preparation workbook.

You fill in the workbook, point the module at it, and it produces the JSON payloads you feed to the VCF Installer and SDDC Manager.

No more hand-writing JSON. No more copy-paste mistakes.

For most deployments it works exactly as intended.

Until you build a management domain that does not run on vSAN.

Problem

Not every management domain uses vSAN.

Some environments run their management cluster on classic block storage. In my case: VMFS on Fibre Channel (FC).

The Planning & Preparation workbook fully supports this. On the Deploy Management Domain sheet the principal storage dropdown offers four options:

  • vSAN-ESA
  • vSAN-OSA
  • VMFS on Fibre Channel (FC)
  • NFSv3

So I selected FC, filled in the rest of the workbook, and generated the Management Domain JSON.

The result looked fine at first glance. Until I checked the storage section:

"datastoreSpec": {

"vsanSpec": {
"datastoreName": "mgmt-fc-datastore01",
"esaConfig": null
}
}

 

That is a vsanSpec.

On a domain that has no vSAN at all.

The module had silently ignored my storage choice and produced a vSAN payload anyway. And it did not stop there: the generated networkSpecs also contained a VSAN network, and one of the DVS specs was dedicated to VSAN traffic.

If you submitted this to the VCF Installer, you would be describing a completely different deployment than the one you designed.

That is exactly the kind of surprise you do not want during a bringup window.

Troubleshooting

Retrieving the storage choice was never the problem.

The module reads it straight from the workbook:

'storageModel' = $pnpWorkbook.Workbook.Names["mgmt_principal_storage_chosen"].Value

 

So the value VMFS on Fibre Channel (FC) was available all along. (Note the leading space. The dropdown list literally stores it that way. More on that later.)

The real problem was in the JSON generation.

Inside New-ManagementDomainJsonFile the datastore section is built like this:

$datastoreSpecObject = [pscustomobject]@{

'vsanSpec' = ($vsanObject | Select-Object -Skip 0)
}

There is no branch for anything else.

No elseif for Fibre Channel. No elseif for NFS.

Whatever storage type you picked, the module always emitted a vsanSpec. The storageModel value was only ever compared against VSAN-ESA and VSAN-OSA to decide how to build the vSAN spec, never whether to build one.

The same assumption was baked into the network and DVS sections. A VSAN network was always added. A VSAN transport was always part of the switch layout.

The module was written with one storage model in mind. And for a long time, that was a fair assumption for management domains.

But the workbook clearly offers more. So the generator should honour it.

Solution

First I needed to know what a correct FC payload actually looks like.

The VCF Installer UI gave me the answer. When you configure an FC-based management domain there, the exported spec uses a completely different structure:

"datastoreSpec": {

"vmfsDatastoreSpec": {
"fcSpec": [
{ "datastoreName": "your-fc-datastore" }
]
}
}

 

So instead of a single vsanSpec object, FC expects a vmfsDatastoreSpec containing an array of fcSpec entries.

With the target shape known, the fix comes down to three things:

  1. Detect the storage type once.
  2. Build the correct datastoreSpec for that type.
  3. Stop injecting vSAN networking when there is no vSAN.

One decision, made once

The whole fix hangs on a single boolean, derived from the storage model and trimmed of that annoying leading space:

$storageModelRaw = "$($instanceObject.vsphereClusters[0].storageModel)".Trim()

$isVsanStorage = ($storageModelRaw -like "*vSAN*")

-like is case-insensitive, so this cleanly matches vSAN-ESA, vSAN-OSA and VSAN-*, while VMFS on Fibre Channel (FC) and NFSv3 correctly evaluate to $false.

Branching the datastoreSpec

The existing vSAN logic stays exactly as it was. I simply wrapped it in a guard and added the FC branch next to it:

If ($isVsanStorage)
{
# ... original vSAN logic, unchanged ...
$datastoreSpecObject = [pscustomobject]@{
'vsanSpec' = ($vsanObject | Select-Object -Skip 0)
}
}
elseif ($storageModelRaw -match "Fibre Channel|FC|VMFS")
{
# VMFS on Fibre Channel (FC)
$fcSpecArray = @()
$fcSpecArray += [pscustomobject]@{
'datastoreName' = $instanceObject.vsphereClusters[0].vsanDatastore
}
$datastoreSpecObject = [pscustomobject]@{
'vmfsDatastoreSpec' = [pscustomobject]@{
'fcSpec' = $fcSpecArray
}
}
}
elseif ($storageModelRaw -match "NFS")
{
LogMessage -type WARNING -message "Principal storage type '$storageModelRaw' (NFS) is not yet supported - datastoreSpec omitted."
$datastoreSpecObject = $null
}
else
{
LogMessage -type WARNING -message "Unknown principal storage type '$storageModelRaw' - datastoreSpec omitted."
$datastoreSpecObject = $null
}

 

Cleaning up the networking

A VSAN network on an FC domain makes no sense.

The VSAN networkSpec is now only added when $isVsanStorage is true.

The DVS specs were trickier. They are built across several VDS-profile branches, and the code relies on fixed array indices ($dvsObject[0], $dvsObject[1]). Touching that during construction is asking for trouble.

So I cleaned it up after the DVS objects are fully built, as a post-step:

If (-not $isVsanStorage)
{
Foreach ($dvsEntry in $dvsObject)
{
If (($dvsEntry.PSObject.Properties.Name -contains 'networks') -and $dvsEntry.networks)
{
$dvsEntry.networks = @($dvsEntry.networks | Where-Object { $_ -ne "VSAN" })
}
}
$dvsObject = @($dvsObject | Where-Object {
(($_.PSObject.Properties.Name -contains 'networks') -and $_.networks -and ($_.networks.Count -gt 0)) -or
($_.PSObject.Properties.Name -contains 'nsxtSwitchConfig')
})
}

This strips VSAN from every switch and drops any DVS that would be left carrying nothing (while keeping the NSX overlay switch, which legitimately has no L2 networks array).

Keeping vSAN Safe

This is the part I cared about most.

A fix for FC is worthless if it breaks the thousands of vSAN deployments the module already handles correctly.

Every change is additive and gated behind $isVsanStorage:

  • When storage is vSAN, $isVsanStorage is $true, the original code path runs untouched, and the cleanup step is skipped entirely.
  • Only when you pick FC (or NFS) does the new behaviour activate.

I proved it two ways.

A diff of the module shows the vSAN statements are textually identical. Nothing was rewritten, only wrapped.

And running the same FC workbook through the old and the new module makes the difference obvious:

==== BEFORE (original module) ====

datastoreSpec : {"vsanSpec":{"datastoreName":"mgmt-fc-datastore01", ... }}
networkTypes : VM_MANAGEMENT, MANAGEMENT, VMOTION, VSAN


==== AFTER (patched module) ====
datastoreSpec : {"vmfsDatastoreSpec":{"fcSpec":[{"datastoreName":"mgmt-fc-datastore01"}]}}
networkTypes : VM_MANAGEMENT, MANAGEMENT, VMOTION

 

Correct storage. No phantom VSAN network. And the vSAN path completely undisturbed.

Bonus: Skipping the Menu

While I was in here, I built a small wrapper so I no longer need the interactive menu at all.

The module already exposes a non-interactive path through -userPromptBypass and -targetFilePath. The wrapper asks for the workbook location, loads the (patched) module, checks that the workbook revision matches, generates the Management Domain JSON, and finally scans the output for any leftover placeholders such as <LATER INVULLEN>.

One command. One JSON file. Fully scripted.

.\New-VCFManagementDomainJson.ps1 -WorkbookPath "F:\pnp\vcf-9.1-planning-and-preparation-workbook.xlsx"

 

Benefits

  • Correct datastoreSpec for VMFS on Fibre Channel
  • No more phantom VSAN network or VSAN DVS on non-vSAN domains
  • Clear warnings instead of silently wrong output for unsupported types
  • The existing vSAN behaviour is provably unchanged
  • Non-interactive generation for repeatable, scriptable runs
  • Built-in placeholder check before you submit anything

Final Thoughts

Credit where it is due: VCF.JSONGenerator is a genuinely useful community module, and the heavy lifting was already done.

The gap was simply an assumption. Management domains have run on vSAN for so long that “storage” and “vSAN” quietly became the same word in the code.

The workbook never made that assumption. It has always offered Fibre Channel and NFS. The generator just was not listening yet.

Fixing it was mostly about respecting a choice the user had already made, and being careful not to disturb the path that already worked.

NFS is the obvious next step. The plumbing is in place, the branch is waiting, and all it needs is a confirmed reference spec.

But that is a story for another post.

Script

The complete wrapper script:

<#
.SYNOPSIS
    Genereert non-interactief de VCF Installer "Management Domain" JSON uit een
    ingevulde VCF 9.1 Planning & Preparation (P&P) workbook.

.DESCRIPTION
    Wrapper rond de VCF.JSONGenerator module. Slaat het interactieve menu over en
    roept de (private) generatiefunctie rechtstreeks aan met -userPromptBypass.

    Het script:
      - laadt een specifieke versie van de VCF.JSONGenerator module
      - controleert of de workbook-revisie (Arkham!B2 / pnp_version_history) past
        bij de versie die de module verwacht
      - bouwt de shared- en management-objecten en genereert de JSON
      - saneert de bestandsnaam zodat placeholders als <LATER INVULLEN> geen crash geven
      - scant de gegenereerde JSON op resterende placeholders en waarschuwt

.PARAMETER WorkbookPath
    Volledig pad naar het .xlsx P&P-workbook. Wordt gevraagd als het ontbreekt.

.PARAMETER OutputPath
    Doelpad voor de JSON. Standaard: <map van workbook>\managementDomainSpec-<domein>.json

.PARAMETER ModuleVersion
    Te gebruiken VCF.JSONGenerator versie. Standaard 9.1.0.1005 (laatste release).

.PARAMETER Force
    Overschrijf een bestaand JSON-bestand zonder te vragen.

.EXAMPLE
    .\New-VCFManagementDomainJson.ps1

.EXAMPLE
    .\New-VCFManagementDomainJson.ps1 -WorkbookPath "F:\pnp\vcf-9.1-planning-and-preparation-workbook.xlsx"
#>
[CmdletBinding()]
param(
    [Parameter(Mandatory = $false)][string]$WorkbookPath,
    [Parameter(Mandatory = $false)][string]$OutputPath,
    # Standaard: de lokale, gepatchte -main module (met FC/non-vSAN ondersteuning).
    [Parameter(Mandatory = $false)][string]$ModulePath = (Join-Path $PSScriptRoot 'powershell-module-for-vmware-cloud-foundation-jsongenerator-main\VCF.JSONGenerator.psd1'),
    # Fallback: geinstalleerde Gallery-versie, alleen gebruikt als ModulePath niet bestaat.
    [Parameter(Mandatory = $false)][string]$ModuleVersion = '9.1.0.1005',
    [Parameter(Mandatory = $false)][switch]$Force
)

$ErrorActionPreference = 'Stop'

# Placeholder-markeringen die erop wijzen dat een veld nog niet is ingevuld.
$script:PlaceholderPatterns = @(
    'LATER INVULLEN',
    'LATER OPBOUWEN',
    'ENTER THE ID OF',
    'ENTER-ESX-THUMBPRINT'
)

function Write-Step { param([string]$Message) Write-Host "==> $Message" -ForegroundColor Cyan }
function Write-Ok   { param([string]$Message) Write-Host "    $Message" -ForegroundColor Green }
function Write-Warn { param([string]$Message) Write-Host "    $Message" -ForegroundColor Yellow }

function Get-CleanFileNamePart {
    # Verwijder illegale bestandsnaamtekens (incl. de < > uit placeholders).
    param([string]$Value)
    if ([string]::IsNullOrWhiteSpace($Value)) { return 'management-domain' }
    $illegal = [System.IO.Path]::GetInvalidFileNameChars() -join ''
    $pattern = '[{0}]' -f [System.Text.RegularExpressions.Regex]::Escape($illegal)
    $clean = [System.Text.RegularExpressions.Regex]::Replace($Value, $pattern, '_')
    $clean = $clean.Trim('_', ' ', '.')
    if ([string]::IsNullOrWhiteSpace($clean)) { return 'management-domain' }
    return $clean
}

try {
    # --- 1. Module laden -----------------------------------------------------
    Write-Step "VCF.JSONGenerator laden"
    Get-Module VCF.JSONGenerator | Remove-Module -Force -ErrorAction SilentlyContinue
    if ($ModulePath -and (Test-Path -LiteralPath $ModulePath)) {
        # Voorkeur: de lokale, gepatchte -main module (FC/non-vSAN ondersteuning).
        Import-Module $ModulePath -Force
        Write-Ok "Module geladen vanaf pad: $ModulePath"
    }
    else {
        # Fallback: geinstalleerde Gallery-versie (LET OP: deze mist de FC-fix).
        $available = Get-Module -ListAvailable -Name VCF.JSONGenerator |
                     Where-Object { $_.Version -eq [version]$ModuleVersion }
        if (-not $available) {
            throw "Gepatchte module niet gevonden op '$ModulePath', en fallback-versie $ModuleVersion is niet geinstalleerd. Installeer met:`n" +
                  "  Install-Module VCF.JSONGenerator -RequiredVersion $ModuleVersion -Scope CurrentUser -Force"
        }
        Import-Module VCF.JSONGenerator -RequiredVersion $ModuleVersion -Force
        Write-Warn "Gepatchte module niet gevonden - fallback naar geinstalleerde versie $ModuleVersion (zonder FC-fix)."
    }
    Import-Module ImportExcel -Force -ErrorAction Stop
    Write-Ok "Actieve versie: $((Get-Module VCF.JSONGenerator).Version) ($((Get-Module VCF.JSONGenerator).Path))"

    # --- 2. Workbook-pad bepalen ---------------------------------------------
    if ([string]::IsNullOrWhiteSpace($WorkbookPath)) {
        $WorkbookPath = Read-Host "Geef het volledige pad naar de P&P workbook (.xlsx)"
    }
    $WorkbookPath = $WorkbookPath.Trim().Trim('"')
    if (-not (Test-Path -LiteralPath $WorkbookPath)) {
        throw "Workbook niet gevonden: $WorkbookPath"
    }
    if ([System.IO.Path]::GetExtension($WorkbookPath) -ne '.xlsx') {
        throw "Bestand is geen .xlsx: $WorkbookPath"
    }
    $WorkbookPath = (Resolve-Path -LiteralPath $WorkbookPath).Path
    Write-Ok "Workbook: $WorkbookPath"

    # --- 3. Workbook openen + versiecheck ------------------------------------
    Write-Step "Workbook openen en versie controleren"
    $pnp = Open-ExcelPackage -Path $WorkbookPath
    try {
        $vcfVersionChosen  = $pnp.Workbook.Names["vcf_version_chosen"].Value
        $pnpVersionHistory = $pnp.Workbook.Names["pnp_version_history"].Value

        if ([string]::IsNullOrWhiteSpace($vcfVersionChosen)) {
            throw "Kon 'vcf_version_chosen' niet uit de workbook lezen - is dit een geldige P&P workbook?"
        }

        # $supportedAutomationVersions wordt door de module globaal gezet.
        $supported = $global:supportedAutomationVersions
        $match = $supported | Where-Object { $vcfVersionChosen.StartsWith($_.version) } | Select-Object -First 1

        if (-not $match) {
            throw "VCF-versie '$vcfVersionChosen' wordt niet ondersteund door module $ModuleVersion."
        }
        if ([int]$pnpVersionHistory -ne [int]$match.supportedAutomationVersion) {
            throw ("Workbook-revisie komt niet overeen.`n" +
                   "  Workbook  (pnp_version_history / Arkham!B2) : $pnpVersionHistory`n" +
                   "  Verwacht  door module $ModuleVersion          : $($match.supportedAutomationVersion)`n" +
                   "  => Gebruik een P&P workbook met revisie $($match.supportedAutomationVersion), of een module die $pnpVersionHistory ondersteunt.")
        }
        Write-Ok "VCF $vcfVersionChosen, workbook-revisie $pnpVersionHistory - komt overeen met module"

        # --- 4. Objecten bouwen (in module-scope, private functies) ----------
        Write-Step "Workbook-data uitlezen"
        $mod = Get-Module VCF.JSONGenerator
        $shared = & $mod { param($p) New-SharedInstanceObject     -pnpWorkbook $p -silent } $pnp
        $mgmt   = & $mod { param($p) New-ManagementInstanceObject -pnpWorkbook $p -silent } $pnp
        if (-not $mgmt -or -not $shared) { throw "Kon de management-/shared-objecten niet opbouwen uit de workbook." }
        Write-Ok "Domein: '$($mgmt.domainName)'  |  instance: $($mgmt.instance)"

        # --- 5. Doelbestand bepalen (met sanering) ---------------------------
        if ([string]::IsNullOrWhiteSpace($OutputPath)) {
            $dir  = Split-Path -Parent $WorkbookPath
            $name = "managementDomainSpec-$(Get-CleanFileNamePart $mgmt.domainName).json"
            $OutputPath = Join-Path $dir $name
        }
        if ((Test-Path -LiteralPath $OutputPath) -and -not $Force) {
            $answer = Read-Host "Bestand bestaat al: $OutputPath. Overschrijven? (Y/N)"
            if ($answer -notin 'Y','y') { throw "Afgebroken door gebruiker." }
        }

        # --- 6. JSON genereren (non-interactief) -----------------------------
        Write-Step "Management Domain JSON genereren"
        & $mod {
            param($m, $s, $out)
            New-ManagementDomainJsonFile -instanceObject $m -sharedInstanceObject $s `
                -userPromptBypass -noHostFingerprints -targetFilePath $out
        } $mgmt $shared $OutputPath

        if (-not (Test-Path -LiteralPath $OutputPath)) {
            throw "Generatie meldde geen fout, maar er is geen bestand aangemaakt op $OutputPath."
        }
        Write-Ok "JSON geschreven: $OutputPath"
    }
    finally {
        Close-ExcelPackage $pnp -NoSave -ErrorAction SilentlyContinue
    }

    # --- 7. Placeholder-scan op de output ------------------------------------
    Write-Step "Controle op oningevulde placeholders"
    $content = Get-Content -LiteralPath $OutputPath -Raw
    $hits = @()
    foreach ($p in $script:PlaceholderPatterns) {
        $count = ([regex]::Matches($content, [regex]::Escape($p), 'IgnoreCase')).Count
        if ($count -gt 0) { $hits += [pscustomobject]@{ Placeholder = $p; Aantal = $count } }
    }
    if ($hits.Count -gt 0) {
        Write-Warn "LET OP: de JSON bevat nog placeholders. Deze is NIET klaar voor indiening:"
        $hits | Format-Table -AutoSize | Out-String | ForEach-Object { Write-Host $_ -ForegroundColor Yellow }
        Write-Warn "Vul de betreffende velden in de workbook in en genereer opnieuw."
    }
    else {
        Write-Ok "Geen bekende placeholders aangetroffen."
    }

    Write-Host ""
    Write-Host "Klaar. Output: $OutputPath" -ForegroundColor Green
}
catch {
    Write-Host ""
    Write-Host "FOUT: $($_.Exception.Message)" -ForegroundColor Red
    if ($_.InvocationInfo.ScriptLineNumber) {
        Write-Host "  (regel $($_.InvocationInfo.ScriptLineNumber))" -ForegroundColor DarkGray
    }
    exit 1
}

 

PowerShell VCF VCF9 Bug vSAN Fibre Channel JSON

Leave a Reply

Your email address will not be published. Required fields are marked *