Given the recent changes in access rules for the Twitter API, the NodeXL team has…
Using NodeXL in the Cloud
Using NodeXL in the Cloud:
Using a virtual machine with Amazon Web Services (AWS) Elastic Compute Cloud (EC2)
Authors:
Devin Gaffney
itsme@devingaffney.com
Arber Ceni
ArberCeni@Smrfoundation.org
Additional material from the Amazon EC2 support site.
About NodeXL
About Amazon Web Services Elastic Compute Cloud
Creating an AWS account
Amazon EC2 instance types
Micro instances
Medium instances
Large instances
Extra-large instances
What instance type to choose? What does it cost?
Creating an EC2 instance
Set the password
Turn off Internet Explorer Enhanced Security
Install NodeXL
About NodeXL
NodeXL Basic and NodeXL Pro
Install NodeXL Basic
Install NodeXL Pro
Stop and Start your VM
Creating an “AMI” image for your VM
Terminate your VM
Launch your VM after termination
Related tools and resources
About the authors
About NodeXL [Home]
NodeXL is an easy to use tool from the Social Media Research Foundation (https://www.smrfoundation.org/) that enables network overview, discovery and exploration within the familiar Microsoft Office Excel™ spreadsheet environment. NodeXL users can collect and analyze data from a range of social media sites. NodeXL is very easy to use since it fits into the existing flow of Microsoft Office Excel and is fairly intuitive. NodeXL collects social network data from various sources and can automatically analyze the web of connections, content and leading contributors.
NodeXL requires Windows™ and Office™ on a PC (not a Mac, Linux, iOS or Android). Since some users do not have access to a Windows computer or lack a PC with enough resources (RAM and CPU), this document will show you how to install NodeXL on a remote PC Virtual Machine that you will run in the Amazon Web Service EC2 Cloud.
Users of most any PC, Tablet, Phone or device with a browser (including Macs, Linux, iOS or Android) can run a “Remote Desktop” client that will allow access to the Windows + Office + NodeXL environment.
About Amazon Web Services Elastic Compute Cloud [Home]
Amazon provides “instances” of virtual computers running a range of operating systems including Windows in the cloud. These virtual machine instances come in varying sizes and capacities and prices. The good news is that a free basic level virtual machine is available (even if this is often too small for medium to large scale network datasets). The free version is large enough to get started with NodeXL and to use sample data sets and exercises. Amazon provides the means of connecting to a remote PC using free Remote Desktop software. Used along with NodeXL Basic the free tier EC2 is a way to get into NodeXL with almost zero cost. Amazon also offers more powerful virtual machines with more CPU, RAM and disk space for various reasonable costs. Once connected to a virtual Windows PC with Office, NodeXL can be installed and used on this virtual machine remotely. Data and results created in the VM can be exported via email or other methods like Dropbox in order to save your work.
This tutorial will walk through the steps needed to go from creating an Amazon account to setting up an EC2 VM and ultimately using NodeXL.
Creating an AWS account [Home]
1. Go to:
2. If you have an amazon account, sign in with the same one you use on Amazon for shopping.
Otherwise, create a new account.
If you don’t have an account:
a. Fill in full name and password then create account.
b. Amazon will ask for credit card information but will not bill you unless you use anything beyond the free tier of service.
c. Next, verify your account. You will be asked to accept a phone call from an Amazon phone number and enter a PIN code in order to complete an identity verification. Once the call comes to your phone, enter the pin and then wait for the automated voice to acknowledge that you’ve entered the correct pin. When you hear this confirmation, hang up and wait for a moment as the verification is registered by the AWS service.
d. Once your account is verified, you will pick a support plan – choose the default “basic” option, which is a good (free) place to start.
e. Finally, you’ll be redirected to the main page for Amazon Web Services (AWS) – in the top right, click the big yellow button that says “Complete Sign Up” to sign in for the first time. You’ll then be prompted for the email address and password that you provided when starting to create the account – sign in and then you’ll be logged in!
Amazon EC2 instance types [Home]
Amazon EC2 provides a wide selection of virtual machine instance types optimized to fit different use cases. Instance types are made up of varying combinations of operating system, CPU, memory, storage, and networking capacity.
Computing resources can be flexibly selected to create the appropriate mix of resources needed for your application. Each instance type includes one or more instance sizes, allowing you to scale your resources to the requirements of your workload.
Micro instances [Home]
Micro instances are the lowest-cost general purpose instance type. These instances are Free Tier eligible which means that you can run this small virtual machine for free. The only micro instance type available at the moment is t2.micro which has 1 vCPU (virtual CPU) and 1GB of RAM. This amount of resources will allow you to analyze only very small networks (hundreds of nodes and edges) with NodeXL.
Medium instances[Home]
Medium instances come with more CPU power and RAM. Usually they will feature 1 or 2 vCPU depending on the instance type and 4GB of RAM. It will allow users to analyze small to medium size networks (low thousands of edges) with NodeXL. These type of instances are not free-tier eligible.
Large instances [Home]
Amazon EC2 large instances feature 2 vCPUs and 8GB of RAM. These instance types are appropriate for analyzing medium to large size networks (up to ten to twenty
thousand edges).
Large instances are the perfect solution for the vast majority of NodeXL use cases.
Extra-large instances [Home]
Extra-large (x – x12large) EC2 instances are the most powerful instances available on Amazon and at the same time the most expensive. They vary from 4 to 96 vCPU and from 16 to 384 GB of RAM. On an extra-large instance we can analyze networks with one or two hundred thousand of edges with NodeXL.
Which instance type to choose? What does it cost? [Home]
There are many options available from Amazon and each one of them can be used depending on the use case. For analyzing small networks or if you are a beginner, micro and medium instances are perfect. If you want to analyze larger networks we suggest using a large EC2 instance type.
For more information about instance types please visit:
https://aws.amazon.com/ec2/instance-types/
Detailed information about EC2 pricing is available here:
https://aws.amazon.com/ec2/pricing/
Creating an EC2 instance [Home]
1. Create a Windows virtual machine – which is required to run NodeXL.
http://console.aws.amazon.com/
2. In the top left click the “Services” button – a drop down menu will display. On the left-most column of options in the drop down you’ll see EC2 under the “Compute” section. Click “EC2”.
5. Click the blue “Launch Instance” button in the middle of the screen. You’re then presented with plenty of options to select the type of server you want. Click on “AWS Marketplace” and browse through the list of available options.
We will choose “Microsoft Windows Server 2012 R2“.
Note: Entries which have a little tag that says “Free tier eligible” can be mounted on a micro instance and are free.
6. Click “Launch”. On the next page, without selecting any of the other options (i.e. keeping it selected on the one with the green tag that says “free tier eligible”), click the blue “Review and Launch” button. On the next page, click “Launch”.
7. Here you’ll be presented with a request to create a PEM key. This is the key that lets you get into the computer once it’s started up and is unique to your account. Choose “create a new pair” from the dropdown, then give it the name of “nodexl_sna.pem”, and finally click “Download Key Pair” – you’ll now have a file called nodexl_sna.pem in your Downloads directory.
Do NOT delete this file under any circumstances, otherwise you will be forced to repeat the entire process of creating a new virtual machine instance (i.e. from step 3 onward), and if you lose the PEM file, any graphs stored on the virtual machine will no longer be accessible, permanently, and there is no way to retrieve the data.
Click “Launch Instance” once you have confirmed that you’ve downloaded the nodexl_sna.pem file, and then you’ll be redirected to another page.
8. We’ve now created an instance of an Amazon virtual machine. After creating the virtual machine, we now need to navigate to it in order to log in. Click the ID of the virtual machine or click “Instances” in the left-hand dropdown, or click the “Services” drop-down then select EC2, then click “Instances” to navigate to the listing of currently running virtual machines. Since we only have one, we’ll have it already selected.
Now you have a virtual computer all set up and ready to use. But you will still need to install software on Mac and other non-PC laptops and computers that will allow access to the remote virtual computer.
Mac users will install Microsoft Remote Desktop, which is available here:
https://itunes.apple.com/us/app/microsoft-remote-desktop/id715768417
Windows users do not need to download any extra application because remote desktop is available by default on Windows machines.
Logging into your new virtual machine
Mac Users
1. Click “view in Mac App Store”, which launches the app store on your machine. Click “Get”, then “Install”, and enter your apple credentials if necessary. This will install the Remote Desktop installation software on your new virtual computer.
2. Once installed, click “Open” in the App Store. This will launch the Remote Desktop software. Once open, feel free to quit the App Store.
3. Login: We need to get some information about the new virtual machine we’ve created on EC2 in order to log in. To do this, with the machine selected in the EC2 Instances list, click the “Connect” button up on the top, and you’ll be prompted to upload the nodexl_sna.pem file. When prompted, navigate to your Downloads folder, then upload the file back to Amazon, and upon completion, you’ll see a button that says “Download Remote Desktop File”. Download the file, then click on the file, and Remote Desktop will automatically connect to the machine you’ve created. Do not exit this panel yet! We need it for the next step.
4. Clicking the downloaded remote desktop icon allows us to log into the machine. It will start the Microsoft Remote Desktop software we just installed.
5. Annoyingly, we will need to enter the very long password that is shown on the prompt. Once we enter the password, the computer will go through a few startup tasks, and then we’ll be done!
Next, we need to personalize the machine a bit to make it easy to use all the time.
Set the password [Home]
1. Click the windows icon in the bottom left, then click “Control Panel”
2. Click User Accounts in the top right
3. Click “Change Account Type”
4. Click Administrator
5. Click “Change The Password”
6. Enter the existing, long annoying password as requested, then enter the new password for both fields.
7. Give it a hint if you want, and then update. You have now reset the password.
Turn off Internet Explorer Enhanced Security [Home]
On many systems it is necessary to configure security in order to access the NodeXL web site and install software.
To do this:
1. Click the windows icon in the bottom left, then click “Server Manager”
2. In the left-hand sidebar, click “Local Server”
3. Under the “Properties” section, find “IE Enhanced Security Configuration” on the right hand side and click the blue “On” link.
4. Turn off for Administrators as well as Users.
Install NodeXL [Home]
About NodeXL Basic and NodeXL Pro [Home]
NodeXL has been split into two versions: NodeXL Basic and NodeXL Pro. NodeXL Basic is positioned as a browser for files created with NodeXL Pro.
In order to run NodeXL Basic, please uninstall NodeXL Pro. Then install the NodeXL Basic application from:
https://nodexlgraphgallery.org/Pages/Registration.aspx
In order to run NodeXL Pro, please uninstall NodeXL Basic. Then install the NodeXL Pro application from http://nodexlgraphgallery.org/Pages/Registration.aspx
Some users have an issue that blocks install of NodeXL Basic after the upgrade to NodeXL Pro – and vice versa.
This issue can be resolved – please go to the following path before installing your preferred version of NodeXL:
C:\Users\[user_profile]\AppData\Local\Apps\2.0
You should see two folders: one named Data and another folder with a strange name. The oddly named (jumbled letters and numbers) folder is where the old NodeXL is stored.
Please delete the folder with the strange name and try reinstalling your preferred version of NodeXL.
You will be able to read any workbooks you have created in older versions of NodeXL in NodeXL Basic.
Creating new network metrics will require use of NodeXL Pro.
Install NodeXL Basic [Home]
1. Visit:
https://nodexlgraphgallery.org/Pages/Registration.aspx
then click “Download” to download the latest copy of NodeXL.
Please select from the various versions of NodeXL.
NodeXL Basic is free but has limited functionality.
NodeXL Pro has a license fee and has a full set of features.
We offer 12 month and 1 month NodeXL Pro licenses.
- Commercial user licenses are available for $749 for 12 months or $75/month.
- Academic staff and faculty user licenses are available for $199 for 12 months.
- Student user licenses are discounted further, $29 for 12 months.
All NodeXL Pro licenses are the same software – only the fee is different based on ability.
>>>>> IMAGE of download screen with BASIC as an option
2. After downloading, click run and walk through the installation.
3. Once complete, click the Windows Start menu icon on the bottom left, and then click the magnifying
glass at the top right to search the machine – type in “node” and you will see that NodeXL Basic Template is a suggested result. Click that to start
NodeXL.
You may also right-click this result and select “Pin to menu” to place NodeXL in your Start Menu.
Install NodeXL Pro [Home]
Go to
https://www.nodexlgraphgallery.org/Registration.aspx
to purchase a license for NodeXL Pro.
Your license is sent to the e-mail address associated with the PayPal account used to purchase the license.
Download NodeXL Pro from:
https://www.nodexlgraphgallery.org/NodeXLSetup/NodeXLProExcelTemplate2014Setup.exe
4. Follow these instructions on your e-mail to activate NodeXL:
To activate your copy of NodeXL Pro please install a unique license file on each machine.
The activation process for NodeXL Pro takes just a few steps:
1. Download the attached file(s) containing your NodeXL Pro license(s) on a secure machine. (The file may currently be in your download directory)
2. Copy a single one of these NodeXL Pro license files (each of these files have an extension of .lic) to the machine that is going to run a copy of NodeXL Pro.
3. Paste the license file (.lic) in any location on the local system. You may place this file in any accessible directory. You may want to create a special folder to hold the NodeXL Pro license
key file – this can be a good way to ensure the file is not moved or deleted.
4. Open NodeXL Pro by searching for “NodeXL” from the START menu in Windows. If NodeXL Pro has not been registered before, or if the NodeXL Pro license file has been moved or damaged, you will be asked to select the file. Use the Browse… option to open a file open dialog and navigate to the location of the license file that was sent to you. Then select OK.
5. Your copy of NodeXL Pro is now licensed and should be operational! Enjoy!
Problems?
Contact info@smrfoundation.org and include the email address used to request the license key, a description of the error, and a description of the computer hardware and software (operating system and Office version).
6. If you have purchased more than one NodeXL Pro license, repeat steps 2-5 for each of the rest of the purchased licenses on the target machines.
If you have trouble installing the NodeXL Pro licenses or running the application, please contact us at info@smrfoundation.org or post questions to our discussion board at
https://www.smrfoundation.org/forums/
Stop and Start your VM [Home]
Virtual Machines can be started and stopped. A stopped machine is lost forever. However, a running virtual machine machine can be saved in a snapshot to a file that can later be restored.
In the navigation pane, choose Instances, and select the instance. Choose Actions, select Instance State, and then choose Stop. If Stop is disabled, either the instance is already stopped or its root device is an instance store volume. In the confirmation dialog box, choose Yes, Stop. It can take a few minutes for the instance to stop.
4. To restart the stopped instance, select the instance, choose Actions, Instance State, Start. In the confirmation dialog box, choose Yes, Start. It can take a few minutes for the instance to enter the running state.
If you stop an EC2 instance no charges apply for the usage of the EC2, however some charges apply for the storage used by the file copy of the stopped EC2 instance.
You can find more information about Stopping and Starting AWS EC2 Virtual Machines here:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Stop_Start.html
Creating an “AMI” image for your VM [Home]
An Amazon EC2 Virtual Machine can be written to a file and saved and shared. An “Amazon Machine Image” (AMI) is a copy of your instance of your virtual machine. It can be useful to store a copy of your virtual machine as a backup of a point in time, or to share a tailored set up with other users.
Note that additional license fees for Microsoft Office and NodeXL Pro may apply to additional Virtual Machines.
1. Right-click the instance you want to use as the basis for your AMI, and choose Create Image from the context menu.
2. In the Create Image dialog box, type a unique name and description, and then choose Create Image.
It may take a few minutes for the AMI to be created depending on the VM size and storage type.
After it is created, it will appear in the AMIs view in AWS Explorer.
To display this view, double-click the Amazon EC2 | AMIs node in AWS Explorer.
To see your AMIs, from the Viewing drop-down list, choose Owned By Me. You may need to choose Refresh to see your AMI. When the AMI first appears, it may be in a pending state, but after a few moments, it transitions to an available state.
Terminate your VM [Home]
You can delete your Amazon virtual machine instance when you no longer need it. This is referred to as terminating your instance.
As soon as the state of an instance changes to shutting-down or terminated, you stop incurring charges for that instance.
NOTE: You can’t connect to or restart an instance after you’ve terminated it. However, you can launch additional instances using the same AMI.
Before you terminate the instance, verify that you won’t lose any data by checking that your Amazon EBS volumes won’t be deleted on termination and that you’ve copied any data that you
need from your instance store volumes to Amazon EBS or Amazon S3.
Open the Amazon EC2 console at
https://console.aws.amazon.com/ec2/
In the navigation pane, choose Instances. Select the instance, and choose Actions, Instance State, Terminate. Choose Yes, Terminate when prompted for confirmation.
For more information about terminating your EC2 instance please visit:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html
Launch your VM after termination [Home]
An EC2 instance cannot be restarted after it has been terminated. However we can launch a new EC2 instance from an image (AMI) we have already created before terminating another instance (the steps to create an image from a VM are explained above).
Open the Amazon EC2 console at
https://console.aws.amazon.com/ec2/
From the Amazon EC2 console dashboard, choose Launch Instance.
On the Choose an Amazon Machine Image (AMI) page, choose a private AMI that you own from My AMIs list
On the Choose an Instance Type page, select the hardware configuration and size of the instance to launch.
Larger instance types have more CPU and memory (explained in earlier sections).
By default, the wizard displays current generation instance types, and selects the first available instance type based on the AMI that you selected. To view previous generation instance types, choose All generations from the filter list.
Note:
To set up an instance quickly for testing purposes, choose Review and Launch to accept the default configuration settings, and launch your instance. Otherwise, to configure your instance further, choose Next: Configure Instance Details.
On the Configure Instance Details page, change settings as necessary (expand Advanced Details to see all the settings), and then choose Next: Add Storage:
On the Add Storage page, you can specify volumes to attach to the instance besides the volumes specified by the AMI (such as the root device volume). You can change the available options, then choose Next: Add Tags when you have finished:
On the Add Tags page, specify tags by providing key and value combinations. You can tag the instance, the volumes, or both. Add at least one tag with key = “Name” and value = so you can name your VM. Choose Add another tag to add more than one tag to your resources. Choose Next: Configure Security Group when you are done.
On the Configure Security Group page, use a security group to define firewall rules for your instance. These rules specify which incoming network traffic is delivered to your
instance. All other traffic is ignored. Select or create a security group, and then choose Review and Launch.
On the Review Instance Launch page, check the details of your instance, and make any necessary changes by choosing the appropriate Edit link.
When you are ready, choose Launch.
In the Select an existing key pair or create a new key pair dialog box, you can choose an existing key pair, or create a new one. For example, choose Choose an existing key pair, then select the key pair you created when getting set up.
To launch your instance, select the acknowledgment check box, then choose:
Launch Instances.
For a detailed explanation of the different options of the Launch dialog please visit:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/launching-instance.html
Related tools and resources [Home]
SocialMediaLab:
https://cran.r-project.org/web/packages/SocialMediaLab/index.html
All code R package for social media data collection and analysis
Gephi:
Tool for visualizing networks
About the authors [Home]
Devin Gaffney
itsme@devingaffney.com
Arber Ceni
ArberCeni@Smrfoundation.org