Linux Containers

Docker Tips n’ Tricks for Devs – #0001 – 3 Second to Postgresql

The easiest implementation of a docker container with Postgresql that I’ve found recently allows the following commands to pull and run a Postgresql server for you.

docker pull postgres:latest
docker run -p 5432:5432 postgres

Then you can just connect to the Postgresql Server by opening up pgadmin with the following connection information:

  • Host: localhost
  • Port: 5432
  • Maintenance DB: postgres
  • Username: postgres
  • Password:

With that information you’ll be able to connect and use this as a development database that only takes about 3 seconds to launch whenever you need it.

Xamarin and I Are Hella Busy Hacking This Week

This week, along with the normal duties of getting everything from SSL working to code slung for account management to intellectual property (what is that exactly ūüėģ )… ¬†this week is going to get hella busy. Here’s a few of the¬†public events and training that I’ll be attending this week along with the normal bike n’ hacking n’ gettin’ shit done.

Shared Code Projects, PCL and Xamarin on 7/8/14 @

Intel JFCC Auditorium
2111 N.E. 25th Ave
Hillsboro, OR

James Montemagno ¬†from Xamarin is coming to learn us the deets on how¬†to create common core code that can run on any or all common platforms. Find out the differences between shared code project, portable class libraries, and simple file linking to share more code on iOS, Android, and Windows. This should be pretty kick ass¬†to help kick OrchestrateExecutive off the ground. There’s a little more info here for the event:¬†http://www.padnug.org/.

Database Stuff that aint RDBMS on 7/10/14 @

I’ll @adron be presenting on database types, what’s available out there outside of the relational and RDBMS world. How to resolve various problems with alternate data solutions for better results, better performance and ways to leap around the hurdles that are sometimes faced with RDBMS use. ¬†More info here:¬†http://www.meetup.com/ssdevelopers/events/176032122/

Xamarin Hands-on-Lab/Hackathon on 7/12/14 @

Montgomery Park

Kelly White¬†@mckhendry¬†has put together a hands-on-lab and hackathon, just a few days later hosting a¬†hand-on-lab working with Xamarin to build apps. I’m going to hit up this event too (then go ride a 100+ kilometer bike ride, anybody up for the ride, ping me?) and sling some code on¬†OrchestrateExecutive.¬†Also¬†a little more info here for the event:¬†http://www.padnug.org/.

There’s more, but these are the top few meets I’ll be attending over the next two weeks. Happy hacking!

In-memory Orchestrate Local Development Database

I was talking with Tory Adams @BEZEI2K about working with Orchestrate‘s Services. We’re totally¬†sold on what they offer and are looking forward to a lot of the technology that is in the works.¬†The day to day building against Orchestrate is super easy, and setting up collections for dev or test or whatever are so easy nothing has stood in our way. Except one thing…

Every once in a while we have to work disconnected. For whatever the reason might be; Comcast cable goes out, we decide to jump on a train or¬†one of us ends up on one of those Q400 puddle jumpers that doesn’t have wifi! But regardless of being disconnected from wifi, cable or internet connectivity we still want to be able to code and test!

In Memory Orchestrate Wrapper

Enter the idea of creating an in memory Orchestrate database wrapper. Using something like convict.js one could easily redirect all the connections as necessary when developing locally. That way development continues right along and when the application is pushed live, it’s redirected to the appropriate Orchestrate connections and keys!

This in memory “fake” or “mock” would need to have the key value, events, and graph store setup just like Orchestrate. With the possibility of having this in memory one could also easily write tests against a real fake and be able to test connected or disconnected without mocking. Not to say that’s a good or bad idea, but just one more tool in the tool chest doesn’t hurt!

If something like this doesn’t¬†pop up in the next week or three, I might just have to kick off this project myself! If anybody is interested please reach out to me and let’s discuss! I’m open to writing it in JavaScript, C#, Java or whatever poison pill you’d prefer. (I’m not polyglot to limit my options!!)

Other Ideas, Development Shop Swap

Another idea that I’ve been pondering is setting up a development shop swap. I’ll leave the reader to determine what that means! ¬†ūüėČ ¬†Feel free to throw down ideas that this might bring up and I’ll incorporate that into the soon to be implementation. I’ll have more information about that idea right here once the project gets rolling. In the meantime, happy coding!

Going Live, Data & Pricing @ Orchestrate

Over the last few months while working on the prototype around Deconstructed I’ve been using the Orchestrate service offering exclusively. With their service around key value and graph store easily accessible via API it was a no brainer to get started building ASAP. Today, that service goes full beta! You can get the full lowdown at the Orchestrate site.

You might recall that I mentioned Orchestrate a while back when they lept into the PIE Class a few months ago. So here’s a few quick thoughts on the release and what Orchestrate is.

The basic premise is Orchestrate provides full-text search, time ordered events, graph, key value storage and a lot more. All of these capabilities are offered via an API that create a product that’s extremely easy to get started. Think about what you’d need to do to get full-text search against a key value setup. Really think about it. Yeah? That’s a lot of steps. With Orchestrate you just sign up and start using it. Think about setting up a graph store and managing it on production systems. Yeah? Lot’s of work once it gets used. Again, just sign up, it’s all there, the graph to the key value to the event series and more. All the NoSQL juice you need located in a single service so you’re not fighting and maintaining multiple databases, nodes or whatever you’re working with.

Sing up. Use.

I will copy one thing from the press release….

  • Ad hoc search queries with Lucene
  • Event and time-ordered storage for activity feeds, sensor data
  • Create and query graph relationships
  • Easy to understand pricing
  • Data export at will – no lock-in
  • Standards compliant data security protocols
  • Daily data backups
  • Bulk data loading
  • Daily and hourly usage monitoring
  • A single, simple interface – JSON data in/out
  • Designed to complement existing databases and MBaaS services
  • Client libraries for Java, Node.js, and Go. More on the way!

Using Orchestrate

There are quotes in the press release, but I’ve got a few myself. I’m working to build out a prototype service that I and Aaron Gray will be releasing soon. Our startup is called Deconstructed, but more on that later. Without Orchestrate my dev cycle would be longer each day, as I battle with maintaining the data sources that I need. Without it I would have spent another 2-3 weeks setting up and staging nosql database technology. All things I didn’t really need to do. I needed to focus on the service, the value that we’ll soon bring to our customers.

It really boils down to this, and don’t get me wrong, I’m a total data nerd. But when it comes to building a product or service, the last thing I want to do is fight with managing the data anymore than I have to. That notion inspired me to write “Sorry Database Nerds, Nobody Actually Gives a Shit” which still holds true. I can’t think of a single business that wants to sit around and grok how an index works in a key value or what the spline of text-search queries is going to be.

Pricing

Pricing is sweet, for many that want to try it out things are free. Prices go up a bit more from there, but if you fall into the pricing you’re doing some business and ought to be rolling in a few bucks eh!

The interesting thing to me about pricing is that they’ve structured it around MOp, which stands for MegaOps. More specifically that’s one million API calls or one million operations.

Summary

If you write code, even a little or if you manage data you should do yourself the service and check out what Orchestrate has built. It’s a solid investment of time. I’ll have a lot more on Orchestrate and how we’re using the service for Deconstructed and more on using the service with JavaScript in the coming months. Keep your eyes peeled and I might even have some Dart and C# magic thrown in there to boot! Check em’ out, until later, happy hacking.

JavaScript

Plotting Good Things in Portland :: pdxbridge.js / WTF Databases /

Several people got together yesterday to start planning things for 2014 in PDX. It ranged from coding workshops to PDX Node to Node PDX to what kind of food to eat at for lunch. Ya know, daily tactical things that come along with the big picture items. ūüėČ

bridge.js badge.

bridge.js badge.

Two things that I want to bring up to the community out there. One is a workshop that I’ll likely lead efforts to organize and the other is something I’ll just call pdxbridge.js for now. The workshop will cover the topics of which and what databases to use for what data and how to implement. The pdxbridge.js project is about determining the raised or lowered state of the bridges here in Portland.

Some of the other projects, workshops and other topics we discussed included getting a workshop put together around unit, integration and testing code from a behavioral, test driven development or other approaches. This workshop we don’t have anyone to teach, but we’d (ok, so I really really would love to attend a workshop on this) really like to find somebody who would be willing to teach a workshop of this sort, with a focus on Javascript as the language. On that same topic however, if you’re into Java, Erlang, Scala, Haskell or others and would like to teach a TDD, BDD or related testing workshop please get in touch with me. We will work on making that happen! Ping me at adron at composite code dot com. ūüėČ

Workshop: Intro to Databases & Data

(Relational, Key/Value, Distributed, Graph, Event Series, etc.)

This is a course I’ll lead and others will work with me on to put something extra useful together. We will then teach the workshop as a group, kind of a team paired programming teaching workshop. If there is anything in particular that you’d like to learn about, any questions that you have about data and usage in applications or otherwise add your two cents on this blog entries comments. Over the next month we’ll be putting together the material and have the course available sometime early this year. So if you’d like to attend, jump in at any time with the conversation or just keep a read here and I’ll have more information about the course as we get it put together.

Let’s Make pdxbridge.js Happen!

The pdxbridge.js project is all about determining if a bridge in Portland is up or down. Right now there are  several bridges that matter, that are on this list;

If we add other information to track about the bridges we might add the other 3 that exist and the new bridge that is being built. however the five listed are the only bridges that have a raised and lowered state, and in one case the Steel Bridge has a lowered, partially raised and fully raised state. As shown on the pdxbridge.js badge I threw together (shown above).

To get involved with pdxbridge.js go add your input on this issue I started to discuss our first meet, plan and hack.

Aside

In Portland, Work With Data? Happy Data Hour…

Happy Data Hour is happening this Thursday, I just stumbled upon it and met the organizers and others. It will be a top group of fun people talking about all sorts of modern data ideas, advances and related topics. In addition, drinks are on the organizers of Keen.io and Digital Ocean. I hear that (and they may be organizers too, that Firebase and Chroma)

I’ll be there with an interest in conversing related to my latest startup project, how data relates to that and generally putting away a drink or two. So please do come and enjoy a good time…. ¬†cheers, and bottoms up!

Review via Calagator and RSVP on Eventbrite.

Sorry Database Nerds, Nobody Actually Gives a Shit…

So I’ve been in more than a few conversations about data structures, various academic conversations and other notions about where and how data should be stored. I’ve been on projects and managed projects that involve teams of people determining how to manage data so that other people can just not manage data. They want to focus on business use and not the data mechanisms underneath. The root of everything around databases really boils down to a single thing – how can we store X and retrieve X – nobody actually trying to get business done or change the world is going to dig into the data storage mechanisms if they don’t have to. To summarize,

nobody actually gives a shit…

At least nobody does until the database breaks, or somebody has to be hired to manage or tune queries or something or some other problem comes up. In the ideal world we could just put data into the ether and have it come back when we ask for it. Unfortunately we have to keep caring for where the data is, how it’s stored, the schema (even in schema-less, you still need to know the schema of the data at some point, it’s just another abstraction to push off dealing with the database), how to backup, recover, data gravity, proximity and a host of other concerns. Wouldn’t it be cool if we could just work on our app or business? Wouldn’t it be nice to just, well, focus on things we actually give a shit about?

Managed Data Systems!

The whole *aaS and PaaS World has been pushing to simplify operations to the point that the primary, if not the only concern, is the business itself. This is a pretty big step in many ways, but holds a lot of hope and promise around fixing the data gravity, proximity, management and related concerns. One provider of services that has an interesting start around the NoSQL realm is Orchestrate.io. I’ll have more about them in the future, as I’ll actually be working on hacking on some code against their platform. They’re currently solving a number of the mentioned issues. Which is great, a solid starting point that takes us past the draconian nature of the old approach to NoSQL and Relational Databases in general.

There has been some others, such as Mongo Labs or such, that have created a sort of DBaaS. This however doesn’t fill the gap that Orchestrate.io is filling. So far almost every *aaS database or other solution has merely been a single type of database that a developer can just throw data at in a single kind of way. Not really flexible, and really only abstracting some manual work, but not providing much additional value add around using the actual data. Orchestrate.io is bridging these together with search, replication and other features to provide a platform on which multiple options are available via the API. Key value, geo, time series and others are all coming together for them nicely. Having all the options actually creates a real value add, versus just provide one single way to do one thing.

Intelligent Data Systems?

After checking out and interviewing Orchestrate.io recently I’ve stumbled into a few other ideas. It would be perfect for them to implement or for the open source community to take a stab at.¬†What would happen if the systems storing the data knew where to put things? What would be the case for providing an intelligent indexing policy or architecture at the schema design decision layer, the area where a person usually must intervene? Could it be done?

A decision tier that scans and makes decisions on the data to revamp the way it is stored against a key value, geo, time series or other method. Could it be done in real time? Would it have to go through some type of processing system? The options around implementing something like this are numerous, but this just leaves a lot of space for providing value add around the data to reduce the complexity of this decision making.

Imagine you have key value data, that needs to be associative based on graph principles, that you must store in a highly available system with pertinent real-time data provided based on those graph relations. A decision layer, to create an intelligent data system, could monitor the data and determine the frequent query paths against the data. If the data is growing old it could move data from real-time to archival via the key value. Other decisions could be made to push up data segments into a cache tier or some other mechanism to provide realtime graph connections to client queries. These are all decisions that would need to be made by somebody working on the data, but could be put into a set of rules to allow for re-allocation of the data via automated mechanisms into better storage options. Why keep old data that isn’t queried in the active in memory graph store, push it to the distributed key store. Why keep the graph data on drive when it can be in memory with correlated keys in a key value in memory store, backed by an on drive key value? All valid decisions, all becoming better understood day by day. It’s about time some of this decision process started to be automated.

What are your thoughts? Pro-intelligent data systems or anti-intelligent data systems? Think it’ll work or is it the wrong approach? Maybe the system should approach some other zenith or axiom point to become truly abstracted and transparent?