Get Virtual Machines names attached to Virtual Network

There is Azure Resource Graph Explorer which is a very useful tool for auditing cloud resources over multiple subscriptions under the same tenant

Nowadays, Azure Portal does not have any view which provides possibility to understand which resource exactly connected to a Virtual Network because most of resources are attached to Vnet over its Network Interface. So, when we look to Connected Devices tab we see only NICs names and not names of resources.

It possible to make more complex views with KUSTO query language and join information about several resources. In our case, to get the list of VMs with associated Vnets, there are VMs, NICs and VNets:

Resources
| where type == "microsoft.compute/virtualmachines"
| project name, vmnics = (properties.networkProfile.networkInterfaces)
| mv-expand vmnics
| project name, vmnics_id = tostring(vmnics.id)
| join (Resources | where type == "microsoft.network/networkinterfaces" | project nicname=(name), vmnics_id = tostring(id), properties) on vmnics_id
| mv-expand ipconfigs = (properties.ipConfigurations)
| extend subnet_resource_id = split(tostring(ipconfigs.properties.subnet.id), '/')
| order by name asc, nicname asc
| project vmname=(name), nicname, vnetname=subnet_resource_id[8], subnetname=subnet_resource_id[10]

How to launch the query in Azure Resource Graph Explorer: https://docs.microsoft.com/en-us/azure/governance/resource-graph/first-query-portal

Terraform workaround: Azure RM template deployment parameters types

If you are using azurerm_template_deployment terraform resource and getting following errors:

  • ‘[parameter]’ expected type ‘string’, got unconvertible type ‘array’
  • ‘[parameter]’ expected type ‘string’, got unconvertible type ‘object’
  • ‘[parameter]’ expected type ‘string’, got unconvertible type ‘int’
  • etc.

Then you are using parameters argument of this resource, so that is possible to pass only string-type parameters with this mechanic.

As an example, the problems begins when you need to pass reference to a KeyVault secret by parameters.

There is a closed issues on AzureRM Terraform provider on GitHub which seems to be impossible to resolve https://github.com/terraform-providers/terraform-provider-azurerm/issues/34

To avoid this error only possible way which I have found it to use parameters_body argument. In this case we will lost any validation by Terraform except validity of parameters JSON but we will able to put any type of parameters. The further validation of parameters will be done by ARM.

Example:

variable "array_variable_example" {
  default = <<EOF
    [ 
        "Element1",  
        "Element2", 
        "Element3" 
    ]
  EOF
}

variable "object_variable_example" {
  default = <<EOF
    {
      "key": {
        "subkey": "value"
      }`
    }
  EOF
}

variable "int_variable_example" {
  default = "1"
}

variable "string_variable_example" {
  default = "string"
}


resource "azurerm_template_deployment" "terraform_resource_name" {
    name                   = "_deployment_${var.environment}"
    resource_group_name    = "${var.resource_group_name}"

    template_body          = "${file("${path.module}/arm/azuredeploy.json")}"

    parameters_body  = <<EOF
      {
            "keyVaultSecretParameter": {
              "reference": {
                "keyVault": {
                  "id": "${var.key_vault_id}"
                },
                "secretName": "${var.secret_name}"
              }
            },

            "objectParameter": {
              "value": "${var.object_variable_example}"
            },

            "stringParameter": {
              "value": "${var.string_variable_example}"
            },

            "arrayParameter": {
              "value": ${var.array_variable_example}
            },

            "integerParameter": {
              "value": ${var.int_variable_example}
            }
      }
    EOF

    deployment_mode = "Incremental"
}

Programmatically create Azure AD Domain Services with Azure CLI

Azure AD Domain Services does not yet very well documented but from existing documentation and Swagger API specification we can find a way for Azure AD Domain Services creation with enabled LDAPS.

Azure AD Domain Services (Azure AD DS, AAD DS) Swagger REST API specification: https://github.com/Azure/azure-rest-api-specs/blob/master/specification/domainservices/resource-manager/Microsoft.AAD/stable/2017-06-01/domainservices.json
az resource create  --subscription [subscriotion-id] \
                    --resource-group [resource-group-name] \
                    --name [managed-domain-name] \
                    --resource-type 'Microsoft.AAD/DomainServices' \
                    --properties <<EOF
                        {
                            "DomainName"  : "[managed-domain-name]" , 
                            "SubnetId"    : "[subnet-id]”,
                            "domainSecuritySettings": {
                                "ntlmV1": "Enabled",
                                "tlsV1": "Disabled",
                                "syncNtlmPasswords": "Enabled"
                            },
                            "ldapsSettings" : {
                                "ldaps": "Enabled",
                                "pfxCertificate": "[pfx-content-inbase64]”,
                                "pfxCertificatePassword": "[pfx-password]",
                                "externalAccess": "Disabled"
                            }
                        }
                    EOF

My issue on GitHub for the documentation update: https://github.com/MicrosoftDocs/azure-docs/issues/40480#issuecomment-540573164

How to work with Azure Blobs when only Azure CLI installed

To avoid Azure Storage account keys usage and give to user the just enough access that is recommended to use Azure AD authentication and RBAC.

To download or read the blob from Storage Account with Private container, user needs at least “Storage Blob Data Reader” role (even if he is an owner of Storage Account resource)

Azure CLI script example:

subscription_id="00000-0000-0000-0000-00000000"
storage_account_name="<storage-account-name>"
container_name="<container-name>"
blob_path="<blob-name>"
output_file_path="<local-file-name>"

az login
az account set -s $subscription_id
az storage blob download  --account-name "$storage_account_name" \
                          --container-name "$container_name" \
                          --name "$blob_path" \
                          --file "$output_file_path" \
                          --auth-mode login

In Linux you have also Python by default and Python is included with Azure CLI installation (with all Azure, Azure AD, Azure Storage Python modules), following Python script can be used to get the similar to Azure CLI result with Device Login:

import adal

from azure.storage.blob import (
    BlockBlobService, 
    PublicAccess
)
from azure.storage.common import (
    TokenCredential
)

storage_account_name  = "<storage-account-name>"
container_name        = "<container-name>"
blob_path             = "<blob-name>"
output_file_path      = "<local-file-path>"


def get_device_login_token():
    # only for example Azure CLI Application ID
    client_id     = '04b07795-8ddb-461a-bbee-02f9e1bf7b46'
    # Your organisation's Tenant ID which used for RBAC for Storage
    tenant_id     = '<tenant-id>'
    authority_uri = ('https://login.microsoftonline.com/' + tenant_id + '/')
    resource_uri  = 'https://storage.azure.com'

    context = adal.AuthenticationContext(authority_uri, api_version=None)
    code = context.acquire_user_code(resource_uri, client_id)
    print(code['message'])
    mgmt_token = context.acquire_token_with_device_code(resource_uri, code, client_id)

    return TokenCredential(mgmt_token['accessToken'])

block_blob_service = BlockBlobService(
        account_name  = storage_account_name, 
        token_credential  = get_device_login_token()
    )

block_blob_service.get_blob_to_path(container_name, blob_path, output_file_path)

If you use your custom application as Python code then service principal must be registered in the tenant of your organisation. Multi-tenant application ID of Azure CLI is used here as an example, in this case we will see the logins from Python script as from Azure CLI. How to create and register multi-tenant application is explained here: https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-convert-app-to-be-multi-tenant

Create a table in PowerShell

Sometimes we need to have a CSV in output of the script. The manual concatenation of string is not a beautiful solution for object-oriented PowerShell. The easiest way to create the table and export it to CSV for me :

$records = @() 

$records += New-Object psobject -Property @{ Column1 = "Value1" Column2 = "Value2" } 

$records += New-Object psobject -Property @{ Column1 = "Value3" Column2 = "Value4" } 

$records | Export-Csv -NoTypeInformation -Path .\table.csv $records | Out-GridView

Check the size of AD Connect MS SQL Express database

MS SQL Express has database size limit of 10 Gb, so how to know when to switch to another edition of MS SQL?

To check current database size of AD Connect you can use following commands:

>"C:\Program Files\Microsoft SQL Server\110\Tools\Binn\SqlLocalDB.exe" info .\ADSync
Name:               ADSync
Shared name:        ADSync
Owner:              NT SERVICE\ADSync
Instance pipe name: np:\\.\pipe\LOCALDB#SHD66A55\tsql\query

>"C:\Program Files\Microsoft SQL Server\110\Tools\Binn\SQLCMD.EXE" -S "np:\\.\pipe\LOCALDB#SHD66A55\tsql\query" -Q "sp_helpdb ADSync;" > ADSync.txt

You can also connect to ADSync database with Invoke-Sqlcmd cmdlet from SQLPS PowerShell module:

Invoke-Sqlcmd -ServerInstance "(localdb)\.\AdSync" -Query 'sp_helpdb ADSync'

Python 3.5 Essentials – 1. Output

When you are trying to learn a new language it is always better if you will see a practical real life result. Let’s try to learn Python from zero and develop a chess game.

The first topic to onboard is Output, because it is the only one way for human to get a result of program execution. For our first programs we will use a text output on screen to console (terminal).

For this series of articles I am using Ubuntu 16.04 where Python 2.7.12 and Python 3.5.2 are preinstalled.  I will write an article to make an instructions for using examples on Windows.

To print some text on the screen we should use print() function in Python. You can create chess0.py text file with following code:

print(" abcdefgh ")
print("8RNBQKBNR8")
print("7PPPPPPPP7")
print("6        6")
print("5        5")
print("4        4")
print("3        3")
print("2PPPPPPPP2")
print("1RNBQKBNR1")
print(" abcdefgh ")

To run this code, open console and type following command:

python3 chess0.py

Now, Python 3.5 will interpret your code and print the chessboard on the screen:

 abcdefgh 
8RNBQKBNR8
7PPPPPPPP7
6        6
5        5
4        4
3        3
2PPPPPPPP2
1RNBQKBNR1
 abcdefgh

Usually any console is using monospaced fixed-width fonts, so, all characters should be well aligned.  But i=even if we can understand the figures it does not looks like a chessboard. To make it more realistic we can use escape sequences. Now you can modify your code to add right background colors for chessboard cells:

print("  abcdefgh  ")
print("8 \x1b[38;5;232m\x1b[48;5;231mR\x1b[0m\x1b[38;5;231m\x1b[48;5;232mN\x1b[0m\x1b[38;5;232m\x1b[48;5;231mB\x1b[0m\x1b[38;5;231m\x1b[48;5;232mQ\x1b[0m\x1b[38;5;232m\x1b[48;5;231mK\x1b[0m\x1b[38;5;231m\x1b[48;5;232mB\x1b[0m\x1b[38;5;232m\x1b[48;5;231mN\x1b[0m\x1b[38;5;231m\x1b[48;5;232mR\x1b[0m 8")
print("7 \x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m 7")
print("6 \x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m 6")
print("5 \x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m 5")
print("4 \x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m 4")
print("3 \x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m\x1b[38;5;231m\x1b[48;5;232m \x1b[0m\x1b[38;5;232m\x1b[48;5;231m \x1b[0m 3")
print("2 \x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m\x1b[38;5;232m\x1b[48;5;231mP\x1b[0m\x1b[38;5;231m\x1b[48;5;232mP\x1b[0m 2")
print("1 \x1b[38;5;231m\x1b[48;5;232mR\x1b[0m\x1b[38;5;232m\x1b[48;5;231mN\x1b[0m\x1b[38;5;231m\x1b[48;5;232mB\x1b[0m\x1b[38;5;232m\x1b[48;5;231mQ\x1b[0m\x1b[38;5;231m\x1b[48;5;232mK\x1b[0m\x1b[38;5;232m\x1b[48;5;231mB\x1b[0m\x1b[38;5;231m\x1b[48;5;232mN\x1b[0m\x1b[38;5;232m\x1b[48;5;231mR\x1b[0m 1")
print("  abcdefgh  ")

Run python3 again:

chess1

Now, it looks nicer but our code became unreadable. How to make it better with help of variables we will cover in the next article.

Congratulations with your first program on Python 3.5!

 

Arrays and objects in ARM Templates

Please, review the article about ARM templates before : Authoring Azure Resource Manager templates.

Before thinking about Nested templates that is possible to imagine a lightweight scenario based on standard ARM template capabilities. Types of variables and parameters in ARM template are not only scalars and they can represent objects and arrays.

Template parameters can take as an input object, array, and secureObject. Variables are type agnostic. Grace to JSON, syntax for those types using is mainly defined by JavaScript Objects and Arrays.

Continue reading “Arrays and objects in ARM Templates”

Test your PowerShell DSC configuration on Azure VM

Troubleshooting and debugging PowerShell DSC configurations could be sometimes very painful. If you are preparing your DSC configuration to use in Azure or even on-premises, you would like to test it in the real environment.

Continue reading “Test your PowerShell DSC configuration on Azure VM”

AMQP Filter for Azure Event Hubs with Apache Qpid Proton

There is no a lot of documentation and examples how to deal with AMQP filters by using qpid-proton (Proton C API).

Deffierent approaches for working with Azure Event Hub are well described by the following link: Using Amqp.Net Lite with Azure Service Bus Event Hubs.

In my practice I’m using qpid-proton 0.8. To set the filter for retriving messages from Event Hub by timestamp offsset:

pn_messenger_t *m_messenger = pn_messenger(NULL);

// ...

// About addresses and filters for Azure Event Hubs: 
// https://amqpnetlite.codeplex.com/wikipage?title=Using%20Amqp.Net%20Lite%20with%20Azure%20Server%20Bus%20Event%20Hub&referringTitle=Documentation
char address[] = "amqps://user:password@host:port/path";
char filter_section[] = "apache.org:selector-filter:string";
// timestamp in milliseconds
char filter_value[] = "amqp.annotation.x-opt-enqueuedtimeutc >= '1422627183000'";

// ...

pn_subscription_t *subsribtion = pn_messenger_subscribe(m_messenger, address);

// get a link and the source endpoint (terminus)
pn_link_t 	*link = pn_messenger_get_link(m_messenger, address, false);
pn_terminus_t	*link_source = pn_link_source(link);
pn_data_t	*link_source_filter_data = pn_terminus_filter(link_source);

pn_data_put_map(link_source_filter_data);
  // add a filter entry
  pn_data_enter(link_source_filter_data);
  // symbol key
  pn_data_put_symbol(link_source_filter_data, pn_bytes(strlen(filter_section), filter_section));
  // described value
  pn_data_put_described(link_source_filter_data);
    pn_data_enter(link_source_filter_data);
    pn_data_put_symbol(link_source_filter_data, pn_bytes(strlen(filter_section), filter_section));
    // filter expression as a string
    pn_data_put_string(link_source_filter_data, pn_bytes(strlen(filter_value), filter_value));
  pn_data_exit(link_source_filter_data);

pn_messenger_start(m_messenger);
	
// ... 
	
// here we'll receive only messages with annotation.x-opt-enqueuedtimeutc >= '1422627183000'
pn_messenger_recv(m_messenger, MESSAGES_COUNT);
	
// ...

You can get binaries of qpid-proton 0.8 as NuGet package which are compiled for processors with IA32 instructions set (I need it for my Intel Galileo Gen2 projects).