Map/Reduce intro


Warning: Trying to access array offset on value of type bool in /homepages/44/d229979316/htdocs/web/wp-content/plugins/slideshare/slideshare.php on line 162

Warning: Trying to access array offset on value of type bool in /homepages/44/d229979316/htdocs/web/wp-content/plugins/slideshare/slideshare.php on line 165

I have been playing with different Map/Reduce tools and examples to acquire the whole of view of this programming to solve different problems. Basically there are several patterns that can be identified when you need to tackle the implementation of some algorithm. I would like to leave some references that have served as inspiration:

In the same way I have prepared some slides compiling some basic examples that I have adapted from the original sources to deliver my own tutorial. I firmly believe that the best way of learning someting is try to explain to others so I hope to extend these slides in the future covering most of the patterns with different examples. At the moment I am going to
leave here the presentation:

Now it is time to play with Impala-Cloudera…I will keep you inform of the progress!

Week #9 in Thessaloniki

Just a few comments for this week (to be completed)….

Reading (pi = persona interest)

Writing, reviewing and researching

  • I am finishing the book chapter about publishing statistical data in RDF
  • I am managing a Special Issue in the Journal of Computers in Industry, Elsevier
  • I just realized that Labra added me in the Acknowledgements part of his work about “Multilingual Open Data Patterns” I am very proud of that! (to be honest I just collaborated in the first presentation with some links and specially through some comments with regards to SKOS-XL). I also suggest to read the paper in which each of the patterns is explained and discussed with excellent examples.

Meetings

  • I have had a meeting with my SEERC colleagues to present my prototype and plan next actions in QoS, etc.
  • I have had a meeting with Michalis Vafoupolus to prepare the Linked Data Cup paper.

Coding and Tools

  • I have implemented a real-time based architecture using the Lambda approach and following some hints from Pere Ferrera. It is not the same algorithm and I am just take the approach to tackle the problem not source code. Next steps include to use RDF as views for batch and real-time layers using SPARQL federated queries (for instance Fedex). The example just takes a Twitter stream using Tweet4J API and counts words presenting the results in a HTML page. Documentation is available here and also the the source code (under development).
  • I have linked to the CPV the public procurement notices from UK, USA and AUS.

Other things

Week #8 in Thessaloniki

Just a few comments for this week (to be completed)….

Reading (pi = persona interest)

Writing, reviewing and researching

Meetings

  • I have had a meeting with my SEERC colleagues to present my prototype and plan next actions in QoS, etc.
  • I have attended a call with the Oviedo Council (planned)

Coding and Tools

  • I have re-implemented my real time word counter of Twitter status using RabbitMQ, Node.js, Storm and Redis (see Demo wiki page).
  • I am going to test the MOA framework (based on Weka) for mining data streams. In this sense I have also found an implementation of MOA in Storm.

Other things

I have Internet connection at home!

Week #7 in Thessaloniki

Just a few comments for this week (to be completed)….

Reading (pi = persona interest)

Writing, reviewing and researching

  • I am reviewing a paper for a Special Isuee of a JCR journal
  • I am finishing the book chapter about publishing statistical data in RDF
  • I have also made the first review of WESOMENDER (we have to work hard to get a good contribution but the expectations are high)
  • I have been invited to be part of the PC of the Special Session “Engineering Tool Integration for Industrial Automation System Development (ETAS 2013)” in conjuction with IECON2013
  • I have joined in the research group “Comercio Electronico en Colombia – GICOECOL” thanks to Luz Andrea RODRIGUEZ ROJAS with whom I will collaborate to empower the use of Open Data in e-Health.

Meetings

Coding and Tools

I have implemented a real time word counter of Twitter status using different tecniques:

  • The classical Observer design pattern
  • The Storm framework, I have reused some examples to implement my own spouts and bolts
  • The Trident framework on the top of Storm, I have also reused some examples of the storm-starter project customizing the code to get a better understanding

Other things

This week I have started the 3-month Greek course and I am very happy because I can now understand some words and read a little bit 🙂 Besides my classmates are from a lot of countries: Bulgaria, Germany, Bosnia, France, Serbia, New Zealand, Italy, Moldova and Russia. It is a GREAT experience.

 

Week #6 in Thessaloniki

Just a few comments for this week….

Reading

Writing and reviewing

  • I have written a paper entitled “Semantic-based QoS management in Cloud Systems: Current Status and Future Challenges “. It is a survey of 25 pages in which I review most of the approaches for QoS in Cloud Computing as well as the main techniques for stream reasoning in the Semantic Web area. It must be the kick-off of the architecture for semantic-based QoS management in Cloud Systems. and I also have the acronym “SEQoS” (remember it because you will hear about it!)
  • I have also managed tasks regarding the special issue in COMIND.

Coding and Tools

I have not made any relevant progress in developing tasks.

Week #5 in Thessaloniki


Warning: Trying to access array offset on value of type bool in /homepages/44/d229979316/htdocs/web/wp-content/plugins/slideshare/slideshare.php on line 162

Warning: Trying to access array offset on value of type bool in /homepages/44/d229979316/htdocs/web/wp-content/plugins/slideshare/slideshare.php on line 165

The last week I have been focused on two main tasks: my presentation at the City College and the submission of a paper. Following the same structure of last weeks I leave some links to the activities I am carrying out:

Reading

Writing and reviewing

  • I have continued with the structure and firts contents of two papers and one special issue proposal.
  • I have managed all the abstracts for the COMIND Special Issue.
  • I have submitted a paper to “Computers and Human Behavior
  • I have made the presentation in the following bullet to the Deparment of Computer Science at City College
  • I have reviewed my previous presentation about MOLDEAS and the new one is supposed to be more didactic such as an “Intro” to Linked Open Data

Coding and Tools

I have not made any relevant progress in developing tasks.

 

Week #4 in Thessaloniki


Warning: Trying to access array offset on value of type bool in /homepages/44/d229979316/htdocs/web/wp-content/plugins/slideshare/slideshare.php on line 162

Warning: Trying to access array offset on value of type bool in /homepages/44/d229979316/htdocs/web/wp-content/plugins/slideshare/slideshare.php on line 165

This week I have made some interesting tasks. Following the same structure of last weeks I leave some links to the activities I am carrying out:

Reading

Writing and reviewing

  • I have continued with the structure and firts contents of two papers and one special issue proposal.
  • I have submitted a contribution to the European Data Forum 2013.
  • I have prepared a presentation (I have to make some changes…)

Coding and Tools

I have not made any relevant progress in developing tasks but I have being refreshing my know-how on Python.

 

Week #3 in Thessaloniki

This week I will be updating this post because I am reading a lot of papers and I need a way to track them. Following the same structure of last weeks I leave some links to the activities I am carrying out:

Reading

I have focused on some interesting subjects Statistics (Bayesian networks), Data Streams, Feedback Control Loops, Autonomous Computing and e-Learning systems (this is just for personal interest). I have started, and finished, the next list of papers and books:

Writing and reviewing

I would like to leave the link to an article about “How to review a paper“, an excellent guide to evaluate your reviews and take into account your responsibilities as reviewer.

Coding and Tools

I have not made any relevant progress in developing tasks but I have being refreshing my know-how on R.

Teaching

I have finished the evaluation of alumni in Health Information Systems and I am very proud of the marks and the work carried out by student during the last months. I have some links of their works building mashups but I prefer do not leave here the links due to privacy issues.

Week #2 in Thessaloniki

Hi all,

This week has elapsed very fast and I have made a lot of things that I leave bellow:

Reading

My main concern in the research is how can I address the automatic computation of a lot of sensors (applications, cloud management platforms, etc.), i.e. how can I monitorize resources? and which the variables to be taken into account are. In this sense I have read some papers from my colleagues at SEERC and other authors:

The main outcome of this work has been an small presentation about how to process Big Data applying the Lambda architecture, more specifically adding semantic to this process. It is a just a proposal and first thinkings but I will do my best to debug and design the whole process.

Writing

I have made some progresses in the article about the experience publishing the “Webindex” as Linked Data and I have also planned the potential articles for this year and their contents.

Coding

Sometimes you feel very motivated to test new tools and frameworks and I am now in this phase:

Teaching

I have finished the evaluation of Health Information Systems in Nursing and Physiotherapy course at the University of Oviedo. They have developed very good works applying Web 2.0 concepts for building mashups in the Health sector, I am very proud of all students.

Administrative Stuff

Here it is where I spent most of the time (and thanks to my colleague Fotis) but I finally got (almost) all the required documentation:

I believe something is missing but, anyway, it is just a summary…