Nasty behavior of ESXi - VMs arent able to mount ISO from certain Datastore

I observed a strange behavior in VMWare ESXi, exactly 6.0U3 Build 5572656 but i am sure many other versions are affected too.

We planned to ONE certain LUN for Scratch-Location, Kerneldumps, VMware Tools and our ISO files, but after some "configuration" changes the ESX servers did not allowed to mount any ISO from this certain datastore anymore.


2017-09-25T08:49:47.872Z| vmx| A100: ConfigDB: Unsetting "ide1:0.clientDevice"
2017-09-25T08:49:47.872Z| vmx| W115: ConfigDB_Set: Cannot make config edit ide1:0.fileName="/vmfs/volumes/58e6316b-7473644d-acaf-2c600c945e0e/DEPOT/ISOS/SCCM_LIVE_X64.iso"
2017-09-25T08:49:47.872Z| vmx| I125: Msg_Post: Warning 2017-09-25T08:49:47.872Z| vmx| I125: [msg.configrules.validate.failed.reject] Invalid value "/vmfs/volumes/58e6316b-7473644d-acaf-2c600c945e0e/DEPOT/ISOS/SCCM_LIVE_X64.iso" for configuration key "ide1:0.fileName".  The value was rejected by rule "No System Files".

I found that that ESXi has a config file with rules and wonder why this datastore is there.

/etc/vmware/configrules

  # /etc/init.d/hostd will sync the lines between the below markers on start.
  # SPECIAL_PATHS_START_TAG
  reject regex_case "^/vmfs/volumes/5971d493-023fb4b0-b2b2-246e965a4370/"
  reject regex_case "^/vmfs/volumes/1b49f9ee-f9cac734-9d40-577e80580578/"
  reject regex_case "^/vmfs/volumes/51c0fee1-3f13fc40-7a38-680ea30b2816/"
  reject regex_case "^/vmfs/volumes/58e6316b-7473644d-acaf-2c600c945e0e/" <-- why is this line here ??
  # SPECIAL_PATHS_END_TAG

Browsing the datastore showed that misconfiguration caused a var and a log directory in the root partition of the datastore, as soon we deleted them the hostd put the full pathname of the scratch location into the configrules file. It seems that the agent look into scratch config and starts searching for a "var" folder on the datastore - and put the folder where it finds "any" var folder into this file.

Ontap Clustermode does memorize beside SIDS also the SamAccountName and seems never to forget it

Update : Issue is gone with Ontap 9.6+

This some nasty behavior I observed on Ontap, if you rename Users or Groups in your active directory c-Dot Ontap will keep them in the CIFS ACL, even after deleting all the caches.

Everything will work but it just bugs when you troubleshoot CIFS access. As it memorize the SID fine you can use the function from my last blog post to set remove and add them, this happens so fast that i did not see any disruption.

$admcdot = New-Object System.Management.Automation.PsCredential('admin',$(ConvertTo-SecureString -String 'huehuehue' -AsPlainText -force)) 

Connect-NcController MyNetappCdotStorage -Credential $admcdot 


function Get-SID_NAME(
    [Parameter(Position=2)][string]$domain=$env:userdomain,
    [Parameter(Mandatory=$True,Position=1)][string]$search,
    [switch]$Local)
{
 if($search -match '\\'){
    $domain=$search.Split('\')[0]
    $search=$search.Split('\')[1]
    }
 if($search -match '^S-1-5-21-'){
    $objSID = New-Object System.Security.Principal.SecurityIdentifier($search)
    $objUser = $objSID.Translate( [System.Security.Principal.NTAccount])
    return $objUser.Value
    }else{
        if($Local){
            $objUser = New-Object System.Security.Principal.NTAccount($search)
            $strSID = $objUser.Translate([System.Security.Principal.SecurityIdentifier])
            return $strSID.Value
        } else {
            $objUser = New-Object System.Security.Principal.NTAccount($domain, $search)
            $strSID = $objUser.Translate([System.Security.Principal.SecurityIdentifier])
            return $strSID.Value
        }
    }
} 


Get-NcCifsShareAcl|?{($_.UserOrGroup -match '^MYDOMAIN') -and ($_.UserOrGroup -notmatch 'admins$') -and ($_.Vserver -notmatch 'mc$') }|%{
     if(-not $(Get-SID_NAME $_.UserOrGroup)){
           Remove-NcCifsShareAcl -Share $_.Share -User $_.UserOrGroup  -VserverContext $_.Vserver
          Add-NcCifsShareAcl -Share $_.Share -UserOrGroup $(Get-SID_Name $_.Winsid) -Permission $_.Permission -VserverContext $_.Vserver
    }
} 

Access a Huawei OceanStor Storage over REST-API per Powershell

So a device which cannot be automated is in my opinion not for big buisness, so here my first steps in dealing with the REST-API which involved a lot of Try&Error. You will need to read the API documentation anyway to know which kind of objects you can put into $HWMGTRessource parameter. If you ask why i put this Pipeline Switch there - it is for later use to "Pipe".

REST-Api Documentation can be found here

V300R003C20: http://support.huawei.com/enterprise/en/doc/DOC1000126989

V300R003C10: http://support.huawei.com/enterprise/en/doc/DOC1000111390


function Start-Huawei_RestSession {
  [Cmdletbinding()]
Param(
[Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=0,Mandatory=$true)]
[String]$HWMGTHostName,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=1,Mandatory=$true,ParameterSetName="HWMGTUser")]
[String]$HWMGTUser,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=2,Mandatory=$false)]
[String]$HWMGTPassword,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=1,Mandatory=$true,ParameterSetName="HWMGTCred")]
[System.Management.Automation.PSCredential]$HWMGTCred,
    [Switch]$ADUser
)
    
    switch ($PsCmdlet.ParameterSetName)
    {
        "HWMGTUser" {
            if($HWMGTPassword){
                $HWMGTCred=New-Object System.Management.Automation.PsCredential($HWMGTUser,$(ConvertTo-SecureString -String $HWMGTPassword -AsPlainText -force))
            } else {
                throw "You musst provide a Password for User $HWMGTUser"
            }
        }
        "HWMGTCred" {
            $HWMGTUser=$HWMGTCred.GetNetworkCredential().UserName
            $HWMGTPassword=$HWMGTCred.GetNetworkCredential().Password
        }
    }


    $body = @{username = $HWMGTUser;
             password = $HWMGTPassword;
             scope = if($ADUser){1}else {0}}

    $logonsession=Invoke-RestMethod -Method Post -Uri "https://$($HWMGTHostName):8088/deviceManager/rest/xxxxx/sessions" -Body (ConvertTo-Json $body) -SessionVariable WebSession
    
    $CredentialsBytes = [System.Text.Encoding]::UTF8.GetBytes(-join("{0}:{1}" -f $HWMGTUser,$HWMGTPassword))
    $EncodedCredentials = [Convert]::ToBase64String($CredentialsBytes)
    
    $headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
    $headers.Add("Authorization", "Basic $EncodedCredentials")
    $headers.Add("iBaseToken", $logonsession.data.iBaseToken)

    return [pscustomobject]@{
HWMGTHostName = $HWMGTHostName
HWMGTCred = $HWMGTCred
DeviceId=$logonsession.data.deviceid
                    WebSession=$WebSession
                    Headers=$headers
                    iBaseToken=$logonsession.data.iBaseToken
                    error=$logonsession.error
    }
}

function Invoke-Huawei_RestMethod {
  [Cmdletbinding()]
Param(
[Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=0,Mandatory=$true)]
[pscustomobject]$HWMGTSession,
    [Parameter(Position=1,Mandatory=$true)]
    [ValidateSet('GET','POST','PUT','DELETE')]
    [string]$Method,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=2,Mandatory=$true)]
[String]$HWMGTRessource,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Position=3,Mandatory=$false)]
[int]$HWMGTRessourceID,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Mandatory=$False)]
[ValidateScript({-not $($_ -inotmatch '^filter=\w*(:{1,2})\w*$|^range=\[\d{1,5}-\d{1,5}\]$')})]
    [string[]]$HWMGTFilters,
  [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Mandatory=$false)]
    [System.Collections.Hashtable]$HWMGTRequestBody,
    [Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True,Mandatory=$false)]
    [Switch]$PipeLine
)

    $URI="https://$($HWMGTSession.HWMGTHostName):8088/deviceManager/rest/$($HWMGTSession.DeviceId)/$HWMGTRessource"
    
    if($HWMGTRessourceID){
        $URI+="/$HWMGTRessourceID"
    } elseif($HWMGTFilters) {
        $URI+="?$($HWMGTFilters -join('&'))"
    }

    if($HWMGTRequestBody){
        $result=Invoke-RestMethod -Method $Method -uri $uri -Headers $HWMGTSession.Headers -WebSession $HWMGTSession.WebSession -ContentType "application/json" -Credential $HWMGTSession.HWMGTCred -Body $(ConvertTo-Json $HWMGTRequestBody)
    }else{
        $result=Invoke-RestMethod -Method $Method -uri $uri -Headers $HWMGTSession.Headers -WebSession $HWMGTSession.WebSession -ContentType "application/json" -Credential $HWMGTSession.HWMGTCred
    }
    if ($PipeLine){
        return [pscustomobject]@{
            Result=$result
HWMGTSession=$HWMGTSession
HWMGTRessource = $HWMGTRessource
HWMGTFilters=$HWMGTFilters
            URI=$URI}
        } else {return $result}
}

So here a example how to report all LUNs .. First you have to start a session

$huasession=Start-Huawei_RestSession -HWMGTHostName "MyChineseStorage" -HWMGTCred $(Get-Credential)

Invoke the REST Command to the Storage

$huarequest=Invoke-Huawei_RestMethod $huasession GET 'lun'

Get what you want from the Result

$LUNTable=$huarequest|Select -ExpandProperty data data|Select NAME,ParentName,WWN,OWNINGCONTROLLER,ALLOCCAPACITY,ID

Delete the Session properly .. will happen itself after 20min

Invoke-Huawei_RestMethod $huasession DELETE sessions|Out-Null