in “major” databases there is sometimes an ETL (Extract,Transform, Load) tool. as DataFrames are the “commonly” used data structure in Python for similar operations (and analysis), you can perform all three functions. That said, i prefer to only do the ‘E’ and ‘L’ as they are “simply” accomplished by built-in functions. The ‘T’ require me to use a for loop and read each row using a file handler, so it’s more “convenient” for me to manipulate the data once it’s imported.

it’s important to note that determining which dataset to use can involve unconscious/implicit bias. therefore in analysis (and offering insights), you need to consider the source: no matter the prevailing “wisdom”, one needs to distinguish between fact and opinion.

here is the updated GitHub repository:

https://github.com/LinsAbadia/Python/tree/master/DataFrames

ironic

October 2, 2019

it was “surreal” last night.  on ABC last night, it featured MoJo (an iconic advertising consultancy and, eventually, a “full-blown” agency)that influenced and shaped Australia.  For those that don’t know, ABC  is a national TV channel that’s mainly funded by the government and disallows all forms of commercialism (including ads) – you can’t even discuss (much less promote) any brands on their shows.

it was so meta.  Admittedly, they have a programme called Gruen that “humorously” critiques ads but this somehow felt different as it impacted Australian culture.  It can be argued that it was simply a mirror and a true insight into the Australian zeitgeist.

While prior to them “proper” American or British accents were used to voice over commercials, authentic Australian enunciation and expressions were instead  used (although they were considered “vulgar” by the “establishment”). i think this “honesty” was greatly appreciated by the Australian public. i now believe that you should never be ashamed of your actual heritage despite what you were “taught” and the constant stream of messages to put you down. i’ve got a feeling that’s why they dislike the old country so much – it is said that POME was originally an acronym for Prisoner Of Mother England.

Hogs (Paul Hogan better known by some Americans as Mick “Crocodile” Dundee) was the human face of a former tourism campaign – i challenge anyone to say it wasn’t effective as Australia moved from the 78th preferred holiday destination by Americans to the 7th.  Prior to him, it was all wildlife (like koalas and kangaroos) and he was even recognised, arguably, as Australian of the Year for its success.  The catchphrase was even adapted to the Yanks: “put another shrimp on the barbie” – as no one Down Under referred to them as “shrimp” but as “prawns”.  They say a real measure of a tourism campaign isn’t just about the foreign visits but also abut how good the locals feel about their country.

nothing to see here

September 7, 2019

i came from my neurophysio the other day and a cab driver asked me, “what’s your problem?’ For a moment, i was tempted to answer: “Nothing. What’s yours?” i said the doctors don’t really know and just kept quiet for the rest of the ride – it would have been unfair of me to “pounce” on his ignorance.

there are systemic vestiges of the sigma of disability – that’s simply the reality.  i laud all those that want to change this but the pragmatist in me knows we are still far-off despite all the strides forward.  Pardon me getting on my “soapbox” but the encounter made me reflect.

The NDIS, although its implementation is very “problematic”, IMHO is very good in principle.  However, the reduction of my allocated funding by tens of thousands in the pursuit of the “almighty” surplus seem like a scenario to “rob Paul in order to pay Peter’,  I’m not an economist but encouraging the disabled community to spend can help “stimulate” the Aussie economy.

Maybe my argument is difficult to accept objectively given i’ve got a “horse in this race’.  Can’t the investigation of an inrease to NewStart be viewed with this lens: who says compassion and fiscal responsibility always need to be on opposite sides.

 

system of a down

February 7, 2019

they showed The Godfather series on free-to-air TV (i rewatched the first two instalments but not the third which i don’t really like.  IMHO, the second film is one of those rare gems when the sequel is as good or better than the original).  I felt bad for the character Michael.  Like Abigail in the film The Favourite (side note: the Academy recognised the Aussie screenwriter), (in my experience) it usually doesn’t really matter if you start out as well-meaning but being part of a “corrupt” system changes you accordingly.

As Trevor Noah talks about apartheid, Hannah Gadsby about marginalisation, and Dave Chappelle about #MeToo, they all point out that these issues are systemic and deeply-ingrained.  Good individuals acting independently are insufficient to effect cultural change – it takes targeted actions (sometimes done in concert) for meaningful results.

i know New Amsterdam is just a tv show (mostly involving trite Hollywood pablum) but occasionally Dr. Goodwin showcases important lessons for us all: he may be over-idealistic at times but, essentially, the key is to understand the underlying structure that causes systemic issues.  It is difficult to improve systems but a good understanding is a vital start.

 

stranger in a strange land

November 9, 2018

we were recently in Japan for my wife’s conference.  she usually doesn’t bring me and my son but my in-laws were also overseas – while I can manage on my own (admittedly, with some difficulty), I can’t look after my son and all his needs.  It was a holiday tacked on to a business trip.  There’s a lot to unpack so where do I begin.

language problems were common so my wife had great difficulty arranging for an accessible room for me.  Once she said “accessible” and got “accessi” as a clarification. Another time, she mentioned with enough space to accommodate a “wheelchair” she was told that there was no “chair” in the room only a “sofa”.  Apparently, they refer to it as “barrier-free” over there but knowing that fact didn’t really help.  Thankfully, she had a relative there that could speak the language.  She facilitated the search but some things were still lost in translation.  We managed to get a room that fit all three of us (instead of the two rooms that seemed at one time, the only option for us).  However, they could only manage a short, plastic stool instead of a shower chair so my wife helped me to sit down so I could shower by myself (and not have to get her wet in the process). To some extent, this reminded me of a few of the challenges that we faced in US hotels.  One of the “accessible” rooms was on the ground floor but there were only stairs entering the hotel – luckily there was a rail i could use to go up and down.  Moreover, my wife had to help me shower.  There was another room were the bathroom was suitable but getting there on my own was problematic: a bed effectively blocked my path.  There too was no ramp at the hotel’s entrance but there were also steps and a rail for me to hold onto.

most of their street cuts were too “high” – so my son and I had to help my wife get me on the pavement as my combination, “lighter”. travel walker/wheelchair doesn’t allow for “self-propel”-tion. And, most lifts could only hold me and my family (did I mention I’m claustrophobic and try to avoid elevators if possible – I know, ironic for someone who has difficulty walking and can’t use escalators or travelators) as well as difficulty getting me out of typically-narrow hallways.  That said, we took trains most of the time (as taxis were really much more expensive but occasionally depending on the destination we didn’t have much choice ) because all platforms were (eventually) reachable via lift(s) (granted some were outside the station, some times you had to take more than one, some escalators could be made wheelchair-friendly with the assistance of staff (a first for me) and it’s not for people who are easily lost/poor at directions) and there were prominent “gaps” on to and off a few trains.  But this was better than our prior experience, as we learned first-hand, there were only a few elevators (only select stations had them and they weren’t always working) on the New York subway.

In 2020, Tokyo will host the Paralympics.  In my mind, that’s both good and necessary.  Usually, hotels have only one or two barrier-free rooms despite UN WHO stating that about 10% of the world’s population identifying as disabled.  Sure, not all of them require mobility or sensory aids but that’s still grossly disproportionate.   We missed the mad-rush for hotels in making rooms more accessible by a month.  Like America, the intention is there.  It only goes to show that it’s only not just an issue of wealth or access to technology but about education.  Originally I was upset  (admittedly, i still sometimes do in very extreme cases but you shouldn’t hold it against people for their ignorance but inform them of the proper “etiquette” (and the often unintended consequences of their actions).  Say what you will about Australia’s shortcomings when it comes to disability but i’m yet to find another country that does it better.

 

TBC

 

(bridge the ) Gap Year

October 22, 2018

i first heard the term ‘gap year’ when I migrated to Australia. It’s supposed to be a respite after Year 12 before college (or as they call it here: university).  A year of ‘rest’ from school is a luxury and culturally antithetical from an Asian, developing economy such as the Philippines.  I do believe that this is beneficial to one’s mindset but maybe this is a result of how formal education is currently structured – maybe having social justice integrated with the curricula is more effective.

Bridge The Gap is essentially a movement to help address the inequalities that exist in Australian society:  it seeks to make Aboriginal & Torres Strait Islander’s outcomes similar to that of the Caucasian population as the disparities are obvious.  Regardless of how you feel about the issue, a more general philosophy of empathy for societal challenges may shift our definitions for success and progress.  There is one I found with some aspects sort of the same but what I’m proposing is non-prescriptive and ‘decontesualised’ making it more applicable to most countries and subject to what’s realistic for the individual.

The idea still needs to be threshed out (and the subject of some blog posts) but I think it’s an idea worth exploring further.

born to run

November 24, 2017

PBS Newshour showed again Part 1 of Jeffrey Brown‘s interview with Bruce Springsteen (originally aired December 19, 2016; the video url: https://www.pbs.org/newshour/show/bruce-springsteen-tackles-truth-song-memoir ). It was meant to be a promo of his memoir but it was much more to me.

He might not be my favourite artist or a technical singer but like he says: he’s learned to ‘inhabit his songs’ which makes his songs more believable.  Moreover, his working-class roots makes him seem authentic and relatable. I don’t pretend to be an expert on him (or his numerous works) but it wasn’t until I heard the original acoustic (and much more slower) version of ‘Born in the USA’ that I thought I understood the lyrics and what that song was truly about.

As he says in his interview and his in his new memoir “I wasn’t modest in the assessment of my abilities. Of course, I thought I was a phony (sic). That is the way of the artist. But I also thought I was the realest thing you had ever seen.” It’s about dichotomy, I guess – existing on two different planes at the same time.  For me, a real artist lives (and exists) with contradictions – they are only human after all.

You can watch Part 2 at https://www.pbs.org/newshour/show/music-medicine-bruce-springsteen