Category Archives: Uncategorized

Building a Dynamic Platform with ASP.NET, MVC and Azure

Software development tooling and techniques are moving at the fastest pace ever.  It seems you can’t get through the week without a new tool, service and way of build software!

The biggest issue I have with learning about these new technologies is one of context – it’s very difficult to learn something unless you have an idea of a problem that the tool/service/methodology addresses.

With this in mind I often find it useful to have an idea or product I want to build.

So, over a series of blog posts I’m going to start building out a new platform, and as I go along, I will use a variety of tools and technologies, and in fact I will often refactor (e.g. rebuild) as we go along.

This last point is quite an important one.  A common approach in a world that moves so fast you need to get products to market in the shortest possible timeframe is to build an MVP – not to be confused with Microsoft Valued Professional, MVP in this instance refers to Minimal Viable Product.

Screenshot 2019-05-31 at 11.14.08

An MVP is the most basic, working, production version of a product that can be built and deployed as fast as possible.  The idea is build fast, fail fast.  If a product looks like it is being picked up you can spend time and money building it out with new features or even rebuilding as required to create a better service. On the flipside if the product bombs you’ve not wasted too much time on it! Generally this is done over a series of iterations – you build the core most important bits first and release them, over time we add to them as requirements demand.

So that’s how I will build my app – I will build services using the most common technologies for speed of development, and then rebuild them using more up to date technologies. We’ll also build out as well as refactoring by adding new functionality and services as we go.

The app itself will essentially be a development platform – think Force.com, MS Dynamics or ServiceNow. I want to be able to create a platform that itself will allow me to build apps within it using no code.

We’ll be building it as a set of services, and plug them up as we go along, thus creating additional functionality.  We’ll start simple and small, and slowly get more complex as we go.

The first set of components we build will be;

  • Data Modeler– be able to define data objects through the system itself, rather than defining your own table schemas.
  • Form Designer– be able to build forms using the models we’ve defined.
  • Workflows– be able to build workflows using a GUI.These will start as simple ‘send and email’, and gradually we will build up more complex options.
  • Security– secure the platform using OAuth and ensure this flows through the system.

These fundamental to the system, and after we have these, we’ll start to look at what other options we can add.  I won’t go into these now because a) I haven’t actually decided! And b) I’d like you give you my audience the chance to make suggestions!

These will be built in layers as follows;

Picture4

We will have a main UI – this would be the end user portal, but then we’ll have separate UIs for each individual service.  The aim here is that each service should be able to run independently, however the platform GUI would seamlessly bring them together and make them look like a single UI.

At the bottom we have a series of databases, and then we wrap these databases in individual services that expose and manage that data.  The UI should be completely agnostic of the underlying database – so if we changed the technology, we should only need to update the service.

Any comms between services would be performed by a message broker – i.e. there should be no direct comms either to other services or the databases themselves, in this way we decouple (i.e. remove dependencies) as much as possible.

So that’s the overview – let’s move on to the first task – building the modeller.

The Data API

Our platform needs to support a dynamic data model.  By this I mean that our models should be defined by an advanced user within our system itself.  We therefore need to build a service that allows us to define a data model as a template.

From a traditional tables and fields perspective we’ll have a Schema as a holder for a set of attributes or elements, which will have 1 or more Schema elements which will define that type of data we are storing.

We will then have a Model object that stores user data, with one or more Model Elements that store the actual field level data.  Our model data will hold intrinsic links to the underlying schema as one without the other is useless.  Thus, in a traditional development sense our data structure looks like this;

Picture3

Lost?  OK let’s run through this concept and compare what we want to do to a normal development project using SQL Server.

Imagine we need to store a customer record.  To do this we’d create a table called Customers.  That table has a column for Id, Customer Name and City (OK this is a REALLY simple database).

It might look something like this;

Customers Table

Field Data Type
Id IDENTIFIER
CustomerName NVARCHAR(MAX)
City NVARCHAR(MAX)

In our platform we won’t build a SQL table to store customers, we will first define the Customer Model in the Schema/SchemeElements tables, and any data for that customer will be stored in a Model/ModelElements table.

Thus, we first create the Customer Schema;

Schema Table

Field Data Type Value
Id IDENTIFIER 1
Name NVARCHAR “Customer”

SchemaElements Table

Field Data Type Value
Id IDENTIFIER 1
SchemaId IDENTIFIER 1
Name NVARCHAR “CustomerName”
DataType NVARCHAR “String”
Id IDENTIFIER 2
SchemaId IDENTIFIER 1
Name NVARCHAR “City”
DataType NVARCHAR “String”

So here in our Schema table we define our Customer Model with an internal SchemaId, and a name which is Customer.  We then create a SchemaElement record that defines the two properties we want to store for the customer – CustomerName and City.  For each we link them back to the Schema table using the SchemaId, and set the DataType to string – note we are using c# context here not SQL syntax (i.e. we say string, not NVARCHAR).

When user’s of our system then start to store data they will store the data in the Model and ModelElements tables.  And because these tables link back to the Schema and SchemaElements tables we can make sense of what that data is.

So as a final visualisation – let’s look at how our actual SQL tables with some actual data in them might look like.

Picture2

And then if we want to query across these tables to bring back a list of customers and their element values, we could write a query like this;

SELECT[Schema].Name asSchemaName,SchemaElements.Name asElement,SchemaElements.DataType,ModelElements.Value

FROM  ModelElements INNERJOIN

SchemaElements INNERJOIN

[Schema] ONSchemaElements.SchemaId =[Schema].Id INNERJOIN

Models ON[Schema].Id =Models.SchemaId ONModelElements.ModelId =Models.Id ANDModelElements.SchemaElementId =SchemaElements.Id

Which would return the following;

Picture1

Therefore, actual data we are recording consists of the data itself and a linked template.  In our app we’ll need a couple of screens – the first is an admin page that allows us to define the model, the second is a dynamic form that wraps itself around a model.

But we’re getting ahead of ourselves.  The first thing we will build is an API that will allow us to build a model or template.

I’ll start by creating a new project in GitHub – I’m actually going to be doing this twice – once for my blog and again for my Youtube channel.

If you’d like to see this series on YouTube check it out here https://www.youtube.com/c/completecoder

And the GitHub link for the code for this BLOG is here

https://github.com/completecoder/Agility_Blog

In the next Lecture we’ll start the actual coding! Don’t forget to sign up to be notified of updates.

How to Learn Programming

I am often asked this question, or at least various variations of it.

First let me say that there is a difference between learning to code, learning a programming language, and learning to program.

 

Learning to Code

Learning to code means learning the syntax of a language such as C# or Java.  It’s about know the basics of code constructs.  I.e. how to declare and use variables for temporary storage of data in memory.  It’s about the basics of how to create loops, decision trees, iterative functions and so in.

Learning to code is what most people think they mean when they ask ‘How do I learn to program’. Learning to code is actually quite straight forward, there are a few basic concepts, and some rules to follow. If it were too complicated computers wouldn’t understand what you’re trying to tell them to do!

Learning a Language

Next is learning a language. There are many programming languages available.  Common one’s are C#, Swift, JavaScript and Python.  There are lots of others – however on the whole the perform the same job – they provide a set of instructions to a computer.  They generally do this in the same way, but each has it’s own nuances in the structure of the code.

But when it comes down to it they all store take in input, temporary data in variables, do something with that data using a combination of decision trees, loops and calculations, then return a result.

So, learning a language is about learning the specific commands that are available in that set. However, learning one language gives you a grounding in any languages, and in fact some – for example C#, JavaScript and Swift – are often very similar.

As languages are often updated the list of commands and functions available to you, and sometimes how you do this, is changing constantly.

Learning to Programming

Finally, we have programming.  To many they think this is a combination of the above two.  However, for me, programming is about the structure and nature of how you address a particular problem using a programming language.

This last aspect is quite possible the most fluid subject.  As the adoption of software development has become more accessible, and by that, I mean you are far more likely to come across someone who can program today as you would have been 20 years ago, then the available patterns, methodologies and technologies has exploded.

Over the past 50 years we’ve gone from relatively simple patterns where by code would literally just be one line after the next, to complex systems involving procedural, functional and object orientated paradigms.

These in turn have led to a range of available patterns and frameworks, with many languages spawning their own.  It’s important to understand WHY the landscape has grown so.

50 years ago, computers were not as capable as they are today.  It is said that your mobile phone is considerably more powerful than the computers that took the astronauts to the moon!  If we tried to write software using the same patterns as we did back then the codebase would become unusable.  And in fact, many high profile failures in software projects stem from how they are managed and the way they have been written.

Monolithic is a term often used to describe these older, bug prone patterns.  Today we have developers to use a range of patterns and methodologies such as Object Orientated Programming, Model-View-Controller, Model-View-ViewModel, Model-View-Presenter, Dependency Injection, Dependency Inversion, Single Responsibility…. The list goes on and on!

However, all these patterns are about achieving a set of ideals including better quality, less bugs, easier collaboration, faster coding and shorter release cycles.  All this in turn results in easier working environments, better estimates, better software and ultimately happier customers.

For me THIS is what programming means, and THIS is the bit that takes time, effort and in truth should be a constant element of learning for any developer throughout their career. The first two bits – coding and language – are actually quite simple and easy.  It’s knowing how to program that makes the biggest difference – and unfortunately, it’s developers with this skill that are in short supply (and high demand).

If you’d like to learn to program properly, please checkout my course ASP.NET MVC Complete Training

 

Are you a Coder, a Hacker or a Developer?

Coder, Hacker or Developer – which do you think best sums you up?  I’ll wager that most would consider themselves a ‘Developer’.

But in my many years experience – from learning the basics of coding, to my present day role of Architect and teacher/mentor – I’ve come to the conclusion that actually the three titles are all part of the career path in software development.

For me the titles perform a very good job of defining the role, but I feel the perceptions associated with each can be misleading.

Coder

Coder’s code.  To code means to string some computer language together in logical structures to perform some function.  That’s it.  And in today’s modern Agile world this is an extremely important skill.  Agile is to some extent about breaking down the work into manageable tasks or ‘User Stories’.  As a User I want to do x.

So a Coder will be assigned a task – write some code that completes a function.

All these functions are eventually strung together to making the whole.  But essentially most tasks come down to coding – hence why it’s the first step of all developers.

Hacker

To some a hacker is cool.  To Business & Government hackers are evil!

But let’s examine the term.  Hacker.  To ‘hack’ into an existing system and, well, bend it to your will.

Yes, in the wrong hands this skill can cause some real damage – but so can ANY tool in the wrong (or inept) hands.

But to me Hackers represent the next level of coding skills.  Hackers can obviously code, but hackers can also look into the base foundations that all coders use day in day out, and enhance them. Hackers are able to get right there into the code and make it do things you wouldn’t normally be able to do – how?  Because Hackers truly understand the underlying system they are hacking.

This is also an incredibly important skill.  Rarely do business requirements fit any standard mould.  I have heard coders say – ‘it can’t be done, the system just won’t allow that’.  The Hacker will see the challenge.  They’ll learn the underlying mechanism and find away to make it do what they need it too.

So don’t ever think hacker is a detrimental – term – for me it’s the next level in becoming a truly greater developer.

Developer

For all their technical ability, Hackers still lack an important skill – Business Acumen.  Developers (in the varied sense) are often viewed is ‘techies’.  But a true Developer understands the business needs – and can draw upon their Coder and Hacker skills to find the most effective, efficient solution to the Business Problem.

The true Developer is the most misunderstood role.  A Developer not only understands the underlying possibilities, they can decipher the business speak and apply their knowledge to the problem at hand.

In this sense ‘Developer’ is often referred to as Senior Developer – and it’s not about age or years of service (although that often helps!) – it’s about having the full skill set.  The most successful developers, the one’s who’ve built billion dollar companies, are the ones who also understand Marketing, Sales, Finance and Leadership – not necessarily in the depth needed to specialise – but certainly enough to carry their own and talk the talk.

Anyone can learn the basic skills to lay some bricks, or plumb some pipes – but a Developer understands how the whole fits together!

Architect

To me an Architect is the natural progression of a Developer – They take the business skills to the next level and help drive the full Software Development Life Cycle (SDLC).

The next level is about seeing things beyond the current project, to the multitude of projects that businesses have to take on. It’s about having the knowledge to see where IT can help and drive the solution BEFORE being asked.

It’s about looking for efficiencies of scale – how sharing code or better still, services, between the different projects can help reduce overhead, development time and quality.

 

All these ‘levels’ are equally important – but if you want to push your career or technical ability – then you need to understand the different roles available – and focus on what’s needed to take you to the next level.

HTML and CSS For Everyone!

Web Development seems to many non-techies as a black art.

A core skill for any web developer is HTML and CSS – the ‘languages’ used to build websites and an increasing number of mobile apps.

But what if I were to tell you then these two skills – HTML and CSS – are actually becoming key for EVERYONE in business?

The world today is centered around communications –email, social media, blogs and more. But to be heard in the noise of mass information your’s needs to standout.

If you regularly send marketing emails, manage a blog or social media account, then HTML and CSS is a must have skill – as only with an understanding of these can you hope to build something appealing and eye catching.

Because HTML and CSS are often considered a ‘programming language’ – many shy away.

Learning the basic core skills to create stunning material using HTML and CSS is actually very easy – if you have learned how to use Word or Excel – then you can learn this!

Just think what you could do if you mastered the basics? Create awesome emails to your customers, tweak your company’s or your personal WordPress site, speak on par with your technical teams, create a stunning personalised resume website!

Business today demands and ever increase skill base – and if HTML and CSS aren’t at the top of the list of things to learn in 2016 – they should be!

To learn more and get a discount off my own HTML and CSS Online course and book click Web Development Essentials – HTML and CSS

The Importance of Flexible Application Design

I don’t normally blow my own trumpet, but this week the team I work with won a prestigious ‘Innovation’ award at Capita Plc for an iPad/ASP.NET/WebAPI solution we have built to address a specific business need.

The application essentially allows building Surveyors to record information about properties using an iPad in offline or online mode, sync that data up to a backend SQL database via a ASP.NET WebAPI service, and then expose the data through an ASP.NET Web Portal.  There’s also the usual Reports and dashboards that managers and such like generally swoon over.

The product itself is a great time saver, it allows surveys to be taken and reported on in a fraction of the time compared to pen and paper, or even an excel spreadsheet (by hooking costs into items that are picked by the surveyor).

As good as the solution is, from a business perspective, what really impressed the judges was how easily it it could adapted to running any kind of survey you want without have to re-code anything.

Let me explain a bit more.

Version 1 of the solution was built for a specific type of survey, and as such the original developer involved built the application with certain settings hard coded.  So for example, in this particular survey they wanted to segregate any one building into Blocks, Levels and Items, therefore to accommodate these business requirements the developer created Block, Level and Item entities.

The backend database used these entities, the front end iPad app used these entities.  And it worked fine.

Specific Entities

But then as these things go other areas of the business saw what had been done and naturally wanted the same thing for them.

Business versus Technical Design

The problem was that was different area of the business wanted something slightly different.  some wanted 4 or 5 levels, some wanted only 1 block, some didn’t really want to record costs but rather just snippets of information such as diary events during the course of a building project.

The original plan was to use the v1 as a core but then re-develop both the backend, and the iPad app for each client.

Now from a technical design point of view this is great.  We get to independently give each client exactly what they want.

However from a business perspective this really wasn’t very good.  You see there are 3 major issue with this way forward.

  • Each new client would take 3-4 months to re-develop
  • Multiple codebases – both front and backend for each client
  • no central location for management

Ultimately these 3 issues really come down to one common business problem – cost.

You see, many times the business would want it NOW.  Or at the very least within a couple of weeks, obviously being told 3-4 months PLUS testing is no good.

Secondly, although some commissions were large value, some were only for a few thousand (as it was only a handful of small properties/locations).  Again at 3-4 months the cost just becomes prohibitive.

Third, with multiple sets of code bases and no central management location, looking after all these different implementations would requirement far more support overhead – and therefore costs.

Start from scratch or accept the negatives?

It is about this time that I got involved with the project.

Immediately I saw the issue but more importantly the solution.

To some this may be obvious, but when you’re a true techie, especially someone who has already invented the wheel once, it’s quite hard to see more flexible alternatives.  After all, what does a techie care about costs?

Now I think I’m quite lazy!  But believe me, this has often been a useful trait.

You see I’m lazy in that I hate boring repetitive tasks,  I also hate having to re-do the same thing again and again.  Once done I want it finished so I can move onto another challenge.

So to me the solution was to have a generic ‘level’ that could recurse itself.  Each level then has a type associated with it, and a parentId.

In this way a ‘level’ can be anything you want.  A building, a floor, an area of ground, a room, a swing – whatever you wan and link it in a parent child hierarchy accordingly.  We can then define WHAT it actually is within a template that is also stored as a template entity.

The iPad app and the WebUI simple interrogated the template to work out what kind of control to use, and thus as the template changed the UI just flowed around it accordingly.

Flexible Entities

So what we can now do is build HOW the app will work within the app itself, without having to rebuild either of the UI’s or the backend.  We also get to keep everything in a central location and report across ALL different survey types.

This is nothing new.  There are LOTS of apps that do this, but its surprising how often this design pattern can be used.

The Business MUST come first

Now, this of course caused quite a heated ‘discussion’ between myself and the original developer.  He (quite rightly) pointed out that such a method would complicate the code more, making reporting a bit more difficult and increase the amount of required processing power.

However, although this is all true, the fact is, as developers we shouldn’t care about such things, it is our job (or at least should be) to bring the best possible value to the business – and if that means we have a harder time of it then so be it.

Now don’t get me wrong – I’m not suggesting we should embrace creating horribly complex code – quite the opposite – if you also apply SOLID patterns to your solutions then you can easily split up this complexity into nice manageable chunks.

And as for the processing power, well not really.  Even our 2nd generation iPads quite happily reconstructed the models according to a template on the fly – and we had some BIG templates with LOTS of data.

Reporting caused a few problems initially, but again Microsoft provide all the tools in the form of Analysis and Tabular services to help out when things got too hairy.

But let’s get back on track.  Eventually I won the discussion 🙂

The resultant solution now means new clients can be accommodate within DAYS, not months.  This has a huge impact on costs and allows smaller project to make use of our system.  It also gives the business confidence that it can bid for new work knowing that if they win (and by being able to do the job faster and therefore cheaper, they often win), we can turn around the new system for them within a few days and let them get on with the job as efficiently as possible.

Because lets not forget – software is about adding more value to the business, either with cost savings or a better proposition.