avatar
Arsalan Anwari
IT Consultant, Data Scientist, Computer Scientist
  • Residence:
    Netherlands
  • City:
    Utrecht
  • Age:
    22
Dutch
English
German
IT Consulting
Data Analysis
Data Engineering
Algorithm Development
Software Development
Parallel Computing
  • Analytical mindset
  • Mathematical & technical
  • Not afraid to be direct
  • Studious and entrepreneurial
  • Agile in new environments
  • Has diverse set of skills
  • Diverse set of certifications

Azure Hybrid Cloud

Description

Club Logistic Services had all their data stored externally on Azure without any local backups. This meant if a datacenter failure occurred at Microsoft all of their data would be lost without anyway to recover from it. Azure allows for geo replication of data, but this too expensive for small or average sized companies like CLS. Normally a local network gateway or a S2S VPN gateway has to be created between Azure and the On-premise network to ensure data is transmitted securely. But CLS sadly has their data stored in multiple databases scattered though different Azure Resource meaning multiple gateways needs to be created or a data warehouse needs to be used which is too expensive.

So alongside creating a local backup of their data I was also tasked with streamlining their data landscape to a single database which could be used for data analytical purposes. More about this can be read on the project page “Mapping CLS data flow”. I opted to use Azure Functions to streamline the communication between the different Azure Resources as this is the only Azure resource which can access and communicate between Azure resources without changing their infrastructure too much.

Azure Functions have different pricing options, but namely the “Consumption Plan” and “Premium Plan”. You can only setup private connections between a On-premise server in the Premium Plan, which is too expensive for a company their size (±1500 euros a month) so the consumption plan was preferred. To get around the limitation set by Microsoft the data needed to be relayed from an external location to the On-premise server. This external pc had two virtual machines. One running a public REST API and the other a VPN server. Encrypted data can be send from a Azure function to this REST API which sends it to a Queue stored in a shared drive between the two VM’s. The other VM can can dequeue messages and send them to the on-premise VPN Access Point through a P2S VPN gateway.

When the data is received on the On-Premise Server it is decrypted and stored in a MySQL database where a PowerBI tool can access it.

  • Order Date:
    15-03-2021
  • Final Date:
    15-11-2021
  • Status:
    Active
  • Client:
    Club Logistics Services
  • Location:
    Veendendaal

Sources:

  • Communication Diagram
  • Client reviews



    © 2021 Arsalan Anwari
    Template author:  Lucidy

    Loading Resume...