Getting ready for PowerShell .NET Notebook

The latest release of .NET Interactive Preview 2 (February 6), which includes .NET Notebook for PowerShell. Remember, this is a .NET Core component that is available cross-platform.

This is great! You can start using notebook file and share it across many systems, both Windows and Linux Operating Systems.

Check out Microsoft blog post on “Public Preview of PowerShell Support in Jupyter Notebooks.”

Before you continue, I suggest to get Anaconda 2019.10 (v4.8.1) installed in your system.

Installing .NET Interactive in Ubuntu

In Windows, just takes a few steps to set it up. For Linux, it takes a few extra steps but still is quick enough to get you started.

For Windows, follow the instructions found at the .NET Interactive page in Github.

For Linux, for Ubuntu 18.04, follow the blog post “Ubuntu 18.04 Package Manager – Install .NET Core“.

Basically, in either operating systems, you install:

  • Install the .NET Core SDK
  • Install the ASP.NET Core runtime
  • Install the .NET Core runtime

After these components are installed, proceed to install .NET Interactive Tools, which will include PowerShell support in Jupyter Notebook.

1. Install the .NET Interactive Global tools with this simple command:

$ dotnet tool install --global Microsoft.dotnet-interactive

2. Then install .NET Interactive “Jupyter” component with the following command:

$ dotnet interactive jupyter install

At this point, in Ubuntu, you will encounter the following known error: (see image)

To resolve the issue, use the text editor to open the ~/.bashrc file to add the path to .NET Tools folder:

$ sudo vim ~/.bashrc
## - Add path to .NET Tools:
export PATH=$PATH:~/.dotnet/tools
:wq
$ source ~/.bashrc

Now, we rerun the command, and this time it will complete without any errors:

$ dotnet interactive jupyter install

To verify that all Jupyter kernel was installed, execute the following command:

$ jupyter kernelspec list

Now, you’re ready to work with PowerShell Jupyter Notebook.

Starting Jupyter Notebook

In Windows, you use any console application to start a Jupyter Notebook session using: DOS, Windows PowerShell, and even PowerShell 7 Preview. Have you to use the Anaconda menu shortcut has provided for running the Windows PowerShell prompt?

Better yet, check my instructions on how to create the “Anaconda Pwsh7 Preview Prompt” shortcut in my previous blog post “ANACONDA AND POWERSHELL WORKING TOGETHER!“.()

Back in Linux, open a bash terminal session.

Now, to start a .NET Interactive Jupyter Notebook session, at the console prompt type the following command:

jupyter lab

At this point, the Jupyter Notebook will open on your default browser (Windows or Linux).

The launcher will show all available components for creating notebook files.

Just pick the notebook kernel you wish to start working… let say “.NET PowerShell.”

Notice that I running the $PSVersionTable in the Notebook that the .NET PowerShell kernel is one release behind the latest update.

Now that I test that my .NET Notebook works, I can save my results for later use.

Please, if you encounter any issues with .NET Interactive/.NET Notebook, post them in their Github repo.

Wait! How can I get PowerShell 7 Preview RC 2 updated in .NET Interactive?

I did post the issue about why I was getting PowerShell 7 Preview RC 1 instead of RC2 and got the answer.

It looks like the initial build of .NET Interactive installation will install version ‘1.0.110801‘, which includes PowerShell 7 Preview RC1.

To get the latest build available with PowerShell 7 Preview RC 2, you need to run the update command:

## - To update tool - use PowerShell 7 Preview RC2
dotnet tool update -g --add-source "https://dotnet.myget.org/F/dotnet-try/api/v3/index.json" Microsoft.dotnet-interactive

Run the “jupyter lab” command again and run again the saved *.ipynb.

And that’s it!  As you can see, this command can get your .NET Interactive installation refreshed with the latest build.

Some exciting features are coming down the pipeline. Stay tuned for more!

Anaconda and PowerShell working together!

Yes! To my surprise, when I completed installing the latest update of Anaconda (Anaconda3 2019.10 (64bit) v4.8.1), I realized they have included the following menu item: “Anaconda PowerShell Prompt (Anaconda3)“. Apparently, this menu item has been added for some time.

So, we can take advantage of this shortcut, especially when we can use this console prompt for working with “PowerShell Notebook.” Please, check out Rob Sewell blog post on the recent update .NET Notebook Preview 2 post about “New .NET Notebooks are here – PowerShell 7 notebooks are here.“.

But, Wait! Let’s take this a little further and get you ready to do some fun.

What’s the main advantage?

The “Anaconda PowerShell Prompt” shortcut is already set to activate Anaconda to be used with Windows PowerShell. There’s no need to do a manual activation by opening a DOS command shell and executing:

c:\> conda activate

Trying to use Python without activating Anaconda, it will give you a message.

The activation will allow you to use Python within Windows PowerShell. Or, just use the shortcut “Anaconda PowerShell Prompt.”

As you probably will notice, this menu item only open Windows PowerShell. So, what about PowerShell Core?

This is probably because of PowerShell Core has multiple versions: PowerShell 6.2.4 (GA) and PowerShell 7 Preview (RC2), both supported by Microsoft.

Would you like to create the Anaconda Pwsh7 Prompt shortcut?

Yes! We can create our own PowerShell Core shortcut. And, here’s how to create the shortcut for Anaconda PowerShell 7 Preview.

First, I will make another copy of the original shortcut and label it “Anaconda Pwsh7-Preview Prompt (Anaconda3)“.

Here’s the original path use the Windows PowerShell shortcut:

%windir%\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -NoExit -Command "& 'C:\ProgramData\Anaconda3\shell\condabin\conda-hook.ps1' ; conda activate 'C:\ProgramData\Anaconda3' "

And, here’s my shortcut modification to use PowerShell 7 Preview:

%ProgramFiles%\PowerShell\7-preview\pwsh.exe -ExecutionPolicy ByPass -NoExit -Command "& 'C:\ProgramData\Anaconda3\shell\condabin\conda-hook.ps1' ; conda activate 'C:\ProgramData\Anaconda3' "

Keep in mind, you will need administrator privileges to create this shortcut in the ProgramData Anaconda menu.

After making all the necessary changes to the new shortcut, we got both Window PowerShell and PowerShell 7 Preview working with Anaconda.

Now go ahead and expand your scripting knowledge!

My Truth with WSL 2 in Windows 10

I’ve seen many blog posts looking for specific information on setting up WSL 2 in Windows 10 and especially, on a virtual machine. But, I always end up a little short and figuring out by myself through trial-and-error.

Microsoft WSL 2 Installation documentation page is helpful for most part. But was meant for a physical installation. ()

Let me shared what I found and hope it serves you well.

My Experience

First, I love WSL (Windows Subsystem for Linux)! It’s a great addition to Windows 10, and everyone should learn how to use it.

To get started, follow the instructions on how to get your WSL 1 Linux Distro installed. And, begin with installing Ubuntu 18.04.

Now, get Docker Desktop (), which can be installed in Windows 10 RTM Build 18363 with WSL 1. For the most part, you can start working with docker containers.

To use WSL 2 with Docker Tech Preview, you need Windows 10 Insider build 18917 (or higher). Execute the following command in either DOS or PowerShell console:

wsl -l -v

If it doesn’t work, then it means you still using WSL 1, and it can’t be set to WSL 2. This might be due to the OS is not a Windows Insider version. In WSL 1, the version ‘-v’ parameter is only available for Windows Insider OS.

Now, If it works, then you’ll get the following response:

prompt, use the following command to change from WSL 1 to WSL 2:

wsl --set-default-version Ubuntu-18.04 2

wsl -l -v

if you still can’t set the WSL distro to version 2, it means you’re not using Windows Insider build.

WSL 2 in Virtual Machine

You need to build a virtual machine with the latest Windows 10 Insider build. If you have tried the previous instructions and didn’t work, then to fix the issue run the PowerShell cmdlet on the Hyper-V host (outside the VM):

Set-VMProcessor -VMName [HyperV-VMName] -ExposeVirtualizationExtensions $true

You can use the Get-VMProcessor cmdlet to verify the changes made to the property “ExposeVirtualizationExtensions.” In this case, should show the value change to “True” as shown below:

Get-VMProcessor -VMName [HyperV-VMName] | Format-List

Make sure the virtual machine is restart after making the changes.

What about setting up WSL 2 on an Azure virtual machine? WSL 2 can’t be set up in an Azure virtual machine. You don’t have access to the Azure parent Hyper-V host to use the Set-VMProcessor cmdlet.

Although the Set-VMProcessor cmdlet is not mentioned in the main WSL 2 installation page, you’ll find it hidden in the WSL 2 FAQ page ().

Remember, this cmdlet is very important if you want to set up WSL 2 on a virtual machine in your physical Hyper-V Host.

The Good Stuff – Docker Desktop WSL 2 Tech Preview

First, make sure all of the above settings are in place. This means that you were able to set WSL 2 as the “Default Version” on your favorite Linux Distro.

Open your favorite console, PowerShell, then verify WSL 2 is set by executing the following command:

wsl -l -v

Then, follow the instructions to install the Docker WSL 2 Tech Preview – “Docker Desktop WSL 2 backend“:

At the time of this post, the download Docker Desktop Edge version should be 2.1.7.0.

Note: If you’re on version 2.1.6.0, upgrading to 2.1.7.0, will fail to start. Ignore it! Then, proceed to “Install Update” to complete installation and reboot.

 

Configure Docker for WSL 2

Although Docker is running in the background, you still need to complete configuring Docker to work in WSL 2.

Continue to follow the instruction from the “Docker Desktop WSL 2 backend – Install” section, and you’re done.

Failure to properly configure Docker to WSL 2, you’ll get the following error:

Now, you can start building and working with Docker containers in WSL 2.

Have fun!

Updating ActiveDirectory module in Windows 10

Do you want to use “ActiveDirectory” module in PowerShell 7 RC.1 in Windows 10? For those who haven’t notice yet, seem like one of the recent updates to Windows 10 RTM Build 1909 will includes the latest version of this module.

“ActiveDirectory” module version 1.0.0.0 will not work in PowerShell 7 RC.1. It will give the following error during the import module process:

ActiveDirectory module imports, but the PSDrive AD: is not created.
Set-Location: Cannot find drive. A drive with the name ‘AD’ does not exist.

To correct the issue, you will need to update this module to version 1.0.1.0.

How to install the updated version?

First, make sure you have installed all of the latest Windows updates. Previous ActiveDirectory module will be on version 1.0.0.0.

To install, look in the “App or remove programs | Optional Features” then look under “Add a feature” for the “RSAT: Active Directory Domain Services and Lightweight Directory Services Tools.

It will replace the previous version with the newer one and will work with PowerShell 7 RC.1.

Remember

To use this module the system need to be a member of a domain, or you’ll get the following error message:

WARNING: Error initializing default drive: ‘Unable to find a default server with Active Directory Web Services
running.’.”

Also, it’s only available for Windows 10 RTM Build 1909, Windows 10 Insider Edition, and Windows Server 2019.

PowerShell 7 Release Candidate Is Here!!

The moment everyone has been waiting for some time is here, PowerShell Release Candidate is available for download. This a “Go Live” release officially supported in production by Microsoft.

Everyone in the Microsoft PowerShell Team, with the help of the community, has done an excellent job with the evolution of this new version of PowerShell. Read all about it on the PowerShell DevBlogs recent post “Announcing the PowerShell 7.0 Release Candidate“.

Make sure to read all previous posts as they perfectly outlined under the “Why is PowerShell 7 so awesome?” section of the release candidate post.

Also, it’s not a bad idea to download the recent .NET 3.1 SDK and check out the updated Docker Core SDK Images.

And, have you try:

1. Windows Terminal – Access all of your Windows Shells from one application.

2. Docker Tech Preview – Get the latest Docker Tech Preview for WSL 2.

3. Out-Gridview – Specially developed to work in PowerShell 7 non-Windows, as well as in Windows OS. (Module: Microsoft.PowerShell.GraphicalTools – PowerShell Gallery)

This is just a few items to keep in mind. It will help you to be a productive DevOps and System Administrator.

At Coders Cafe: PowerShell – Introduction to SQL Server Containers

I’ll be presenting at the South Florida .NET User Group Coders Cafe on Tuesday, August 8th, 6:30 PM. Location: Cendyn Spaces, Boca Raton.

Topic: PowerShell – Introduction to SQL Server Containers

Description: This session will be covering the basic of working with Containers and PowerShell Core. We’ll be taking the steps of creating a SQL Server 2019 container in an Ubuntu 18.04 Linux system. Then, will be using PowerShell Core to connect to the SQL Server containers to extract information.

  

Interested in attending this session, click here to register.

Using Linux dpkg packager to install PowerShell 7 Preview in Ubuntu 18.04

Just another way to install PowerShell Preview beside using “apt” or “snap”.  As in this sample, you don’t need to register the package repository.

Get the Preview link

First, look under the release documentation and search for the deb package. In my case I’m install the amd64 version.

Then, right-click on the “powershell-preview_7.0.0-preview.2-1.ubuntu.18.04_amd64.deb”, and select “Copy link address“.

This will copy the following link address:

https://github.com/PowerShell/PowerShell/releases/download/v7.0.0-preview.2/powershell-preview_7.0.0-preview.2-1.ubuntu.18.04_amd64.deb

Download the Preview

Now, I go back to my linux machine and open a terminal session, and I make sure to change directory to the “Downloads” folder.

cd Downloads

Then, I type the following command and the link address:

wget https://github.com/PowerShell/PowerShell/releases/download/v7.0.0-preview.2/powershell-preview_7.0.0-preview.2-1.ubuntu.18.04_amd64.deb

Installing the Preview

Now, I’m ready to install the preview using the dpkg package installer executing the following command:

sudo dpkg -i powershell-preview_7.0.0-preview.2-1.ubuntu.18.04_amd64.deb

Now, we can start working with PowerShell.

In Summary

You can pick and choose the best way to install PowerShell. So, it really takes a few lines get the PowerShell Preview installed quickly.

Reference

Keep learning more PowerShell!

PowerShell Core – Working with Persistent Disk Storage in Docker Containers

This quick blog post will hope to give you a heads up in how to work with container(s) disk data. It’s a known fact that container(s) storing data will not persist if the container is removed. Yes! If you build a container to store your data, it will be gone.

Containers are perfectly suited for testing, meant to fast deployment of a solution, and can be easily deployed to the cloud. It’s cost effective!

Very important to understand! Containers disk data only exist while the container is running. If the container is removed, that data is gone.

So, you got to find the way to properly configure your container environment to make the data persist on disk.

Persisting Data

There are *two quick way to persist data when working with container(s):

1. Create a docker volume.
2. Or, use a local machine folder area.

*Note: There are other solution to help with persisting data for containers, but this a good starting point.

I’m using the docker command line for now. Later, I will be creating some blog post about using Docker Compose and Kubernetes.

I love to use PowerShell Core with Docker command line!

Docker Create Volume

Using docker command “docker volume create <nameofvolume>” will create the volume to help persist data on your local machine.

docker volume create MyLinuxData

Use the following docker commands to check your newly created volume:

* To list all existing docker volume(s):

docker volume ls

* To check “inspect” a docker volume(s) to provide detail information:

docker volume inspect MyLinuxData

Using the “docker volume inspect <VolumeName>.” command line, it will show the volume mount location:

“Mountpoint”: “/var/lib/docker/volumes/MyLinuxData/_data”,

In this case, the mount location is on the Linux box under the Docker Volumes folder. This means all data can persist on you local machine.

Local Machine Folder

This option seems straight forward as there’s no need to create a Docker Volume. Just use the ‘-v’ switch in the Docker Run command line.

In the following command line I’m activating the Docker container with previously configured Microsoft SQL Server instance. I include the ‘-v’ switch to mount a folder on my local machine.

docker run -p 1455:1455 -v /home/maxt/TempSQLBackups:/home/TempSQLBackups --name sql2k19ctp23_v02 -d sql2k19_ctp2.3_sandbox:CTP2.3-Version02

Notice in this case, to verify that my SQL Server container has mount to my the local machine folder, I can execute the following command:

docker exec -i sql2k19ctp23_v02 ls /home/TempSQLBackups

Using “docker exec -i <containerid/name> ls <containerfolderlocation” will display the results of all the files back to the screen. Now, anything you add to that local folder will be accessible to the container.

Summary

This is a good starting point when learning how to work with Docker data in containers. You’ll still go thru trails-and-errors while learning how to build container images, and make data persist for your application. But, it’s much faster and easier to rebuild images. This is one of a most to learn technology.

References

Check out the following blog post as it help me understand about “Persistent Storage”:

PowerShell Core – Updating your SQL Server Linux Docker Containers Images

In this post I’ll be covering how to install some needed components, how to commit the changes, and create a revised images for deployment.

In recent event and meetings, I’ve been talking about how to work SQL Server Linux Containers Docker images. As these images get your container up-and-running quickly they lacks some tools that may be useful to complete the SQL Server configuration.

What’s missing?

The SQL Server images contains a small footprint of Linux Ubuntu 16.04 Operating System (OS) and is meant for quick deployment. The OS side the container need to be kept updated regularly.

At the same time, when you starts exploring inside the container, there still missing components you may want to use:

  • vim – for editing text files.
  • ifconfig – to check your network interfaces.
  • ping – to check IP-Address can be reachable across the network.
  • curl – for transfering data.

So, after you pull the docker image, create the container using “docker run …“, and then get to the container Bash session by using “docker exec -it …“. Remember the bash session only get you to the “root” level as there’s no users set on these containers.

## - First time setup: (for "server:2019-CTP2.2-ubuntu" and )
docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=$SqlPwd01A' -e "MSSQL_PID=Developer" -p 1433:1433 --name sql2k19_CTP2.3 -d mcr.microsoft.com/mssql/server:2019-CTP2.3-ubuntu;

## - Display all active containers;
docker ps -a

At this point make sure the active container status should be in “Up” status. Now can proceed to update the container.

Installing Missing Components

To have access to the container we use the “docker exec …” command.  This command will allow to get access to the container “root” prompt.

## - Configuring your container:
docker exec -it sql2k19_CTP2.3 bash

The first thing I would suggest to do, execute the following to commands:

## - Updating OS:
apt update

apt upgrade

Notice if you try to execute: vim, ping, ifconfig, and curl are not installed in the container images.

Let’s proceed to install these component by executing the following command:

## - Installing additional components:
apt-get -y install \
curl \
vim \
iputils-ping \
net-tools \
powershell-preview

Also, it’s a good idea to create a Downloads folder in case to install other application(s).

## - Create Downloads folder in root:
mkdir Downloads
chmod 755 Downloads

Notice that PowerShell Core Preview was included with the other missing components.  PowerShell has become a great tool to have in a Linux environment.

PowerShell Core SQLServer Module

Although, this is optional but it doesn’t prevent you to include PowerShell Core Preview 6.2.0-RC1 with the SqlServer module which included the “Invoke-Sqlcmd” use by many administrator.  This is a great module to have in a SQL Server container image.

So, from the “root” prompt in the container open PowerShell Core Preview, then proceed to install the SqlServer module preview version 21.1.18095.

## - Open PowerShell Core:
pwsh-preview

## - Install SqlServer module preview:
Install-Module SQLServer -AllowPreRelease

This completes the essential for using PowerShell to help managing a SQL Server instance(s).

How About Anaconda?

We could install the latest version of Anaconda with Python 3.7 in our SQL Server container image.

## - Change directory to Downloads folder:
cd Downloads

## - Download Anaconda with Python 3.7:
wget https://repo.anaconda.com/archive/Anaconda3-2018.12-Linux-x86_64.sh

## - Install Anaconda with Python 3.7:
bash Anaconda3-2018.12-Linux-x86_64.sh

This will give us the ability to test Python scripts within the container.

Testing installed Components

We need to verify that all previously installed components are working. Go back to the container “root” prompt, and to execute the commands:

ifconfig
ping 127.0.0.1
vim ~/.bashrc
pwsh
sqlcmd

Now, executing “sqlcmd” command line will not work unless you add the path to the executable to the “root” ~/.bashrc file:

## - Need to include the path to SQLCMD command:
echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc

## - Refresh ~/.bashrc:
source ~/.bashrc

## - Run Sqlcmd command:
sqlcmd -L localhost -U sa -P 'sapwd'
> select @@version
> go
> exit

This is a good indication that our *SQL Server container is active. And, now we got all missing components installed.

Now, we need to make sure we don’t lose out changes.

Creating your own SQL Server Docker image

This is an important step so you won’t lose the changes already made to the container.  Below are the brief step to follow:

## - Commit the container changes: (repository name must be lowercase but Tags are OK with uppercase)
## -> docker commit "<Get-Container_ID>" "<Image-name>":"<TAG name>"

docker commit "<Get-Container_ID>" sql2k19_ctp2.3_sandbox:CTP2.3-Version01

## - List images included the committed ones:
docker images

## - Stop Image before the Save step:
docker stop sql2k19_CTP2.3
docker ps -a

## - Save docker updated image:
docker save -o ./Downloads/sql2k19ctp23_sandboxVer01.tar sql2k19_ctp2.3_sandbox

The “docker commit …” command, you’ll provide both the image-name (all lowercase) and a TAG name (uppercase allowed). You can be creative in having an naming conversion for you images repositories.

It’s very important to save images after doing the commit. I found out that having an active container would be useless without an image.  As far as I know, I haven’t found a way to rebuild an image from an existing container if the image was previously removed.

Summary

Hope this brief run down on working with SQL Server Docker container images will get you started with modifying existing images for quick deployment.

One thing to keep in mind!

  • The SQL Server Container memory need to be 4GB minimum.
  • In Windows, if your’re using non-Hyper-V virtualization tools such as Virtualbox, the virtual machine memory need to be change to 4GB.
  • Also, when you are creating images, the virtual machine disk size default is 20GB. This may need to be increase unless you keep cleaning/removing images to make room.

Just layout what you need, commit, save and deploy your docker solution in your environment.

Keep learning about this amazing technology!

 

PowerShell Core – How to install SQLServer Module (Preview) in WSL – Ubuntu 18.04

You all have been following Aaron Nelson blog post on Invoke-Sqlcmd availability Cross-Platform in the SqlServer module then you all are probably have proceeded to download the PowerShell SqlServer.

At the same time, on March 5th,  PowerShell Core 6.2.0-rc1 (Release Candidate) was made available for download.
Go and get it!

The thing is, in order to use the Invoke-SqlCmd cmdlet, you need to use PowerShell Preview version 6.2.0-rc1 (or greater).

Now, SqlServer Module can be easily installed in all platforms but I found out that it won’t install in Windows 10 WSL Ubuntu 18.04.

So, What the issue with Windows 10 WSL – Ubuntu 18.04?

Normally, when working with modules in PowerShell Core, I always use the following Cmdlets: Uninstall-Module to remove the module and then Install-Module with the “-AllowPrerelease” parameter. This will work flawlessly, but I found out that it won’t installed it in WSL – Ubuntu 18.04.

I don’t know why but it was installing the non-preview version SqlServer module 21.1.18080.   So, the following PowerShell Core command line will force the installation of SqlServer module version 21.1.18095-Preview.

Install-Module sqlserver -RequiredVersion 21.1.18095-preview -AllowPrerelease -Force

Now, we can start writing PowerShell Core SQL Server scripts in our Windows Subsystem for Linux – Ubuntu 18.04.