Proposal for converging Semeoz work and Transformap wiki of maps

atlassing
visualisation
publication
datacuration
linkingdata
Tags: #<Tag:0x00007fc001622258> #<Tag:0x00007fc0016220c8> #<Tag:0x00007fc001621f88> #<Tag:0x00007fc001621e20> #<Tag:0x00007fc001621c90>

(Gualter Barbas Baptista) #28

How is the current state of development of this project? Is there any place beyond this thread, where we can follow and interact with the plan and ongoing development?


(Maïa Dereva) #29

Hi @gandhiano : the project is starting slowly. I created a kanban here : https://wekan.indie.host/b/5dgN6K4TkK8ixrDRn/semeoz-transformap
And I wait for @toka’s news to decide who work on what.


(Maïa Dereva) #30

One of my unanswered question is : if I create thumbnails for Transformap maps, where can I upload them in the wiki? Is it possible to add a specific field?


(Jon Richter) #31

You can upload files, like images, at https://wiki.transformaps.net/wiki/Special:Upload
Then they will appear in https://wiki.transformaps.net/wiki/Special:ListFiles

The form for entering new mappings, known from https://wiki.transformaps.net/wiki/Form:Datasource, can be edited at https://wiki.transformaps.net/edit/Form:Datasource. Someone knowledgible with this wiki’s syntax could probably be able to add such a function there. I don’t know how this will be reflected in the Edit with Form function of the individual entries, as in https://wiki.transformaps.net/index.php?title=Degrowth_Map&action=formedit for https://wiki.transformaps.net/wiki/Degrowth_Map

Can you add my newly created user almereyda to that board? I sense toka cannot contribute too much attention for timely response right now.


(Maïa Dereva) #32

Thank you for the answers @almereyda

As i’m not “knowledgible with this wiki’s syntax”, I can start with reviewing http://wiki.transformaps.net maps entries to filter out entries that aren’t really maps.

I can also create and upload thumbnails with with “intelligent” files names. For example, for the page https://wiki.transformaps.net/wiki/2000_Acres, I’ll upload the file 2000_Acres.jpg
Then somebody could add them easily?

I invited you on the kanban.


(Jon Richter) #33

@toka Do you feel you can support us here with the technical details?


(Maïa Dereva) #34

By the way, I see we have to send an invoice after 15. of september: how do we finally split the 500€ budget for this project?


(Maïa Dereva) #35

It seems that without @toka the project can not really start, no?


(Jon Richter) #36

I am thinking about this. We saw Semantic MediaWiki can use the same database system which we just set up for our Wikibase query service. Doing so would make all data inside immediately available as Linked Open Data.

I have made the experience it is best to talk with @toka personally.
Right now, as I don’t see the im- and export dynamics sufficiently described, I am still more with creating an infrastructure that allows dynamic generation of thumbnails.

Else there are other ways to get the data out, which @species and me recently unveiled:


(Maïa Dereva) #37

Hi @almereyda
I agree that it would be a better thing to work on a technical solution to connect the wiki and Wordpress but I don’t think it is the point of this “little” project for the moment. @alabaeye, what do you think ?
BTW, I can’t really help on this point. Maybe I can look for the API of Semeoz Worpress (I was told that WP has a native API now), but I don’t know how to use it as I’m not a developper.

About the thumbnails, I also agree it would be great to develop a dynamic solution but I’m not sure it’s possible to center the thumbnails on the maps when there are embebed in a website. When it’s an OSM map or a google map, it’s easy; but sometimes the maps are inside a site, sometimes very low on the page. Having thumbnails of the entire website wouldn’t be nice…

Anymay, if nobody tell me not to do, I can start with reviewing the maps of the wiki and make a list of the links that are not really maps… At least it can be useful :slight_smile:


(Adrien Labaeye) #38

hi @Maia,
Yes the idea was just a manual “synchronization” so to say and creation of thumbnails on the wiki.
I actually already went through all the entries and checked that all that are tagged as “maps” are actual maps. Some may be in development though, but this is also tagged.
Other than that I can’t help much. :confused:


(Maïa Dereva) #39

Thank you for the answer @alabaeye !

It’s great that you checked the entries, I checked this in the kanban.

By the way, I wanted to modify https://wiki.transformaps.net/wiki/2000_Acres (Website Expired) but I still can’t create an account on the wiki, I always receive the message “Incorrect or missing CAPTCHA.” :frowning:

Well, in order to go on this project, I will create all the thumbnails, and I think I can’t help anymore after that…
Work in progress will be here: http://semeoz.info/wp-content/uploads/transformap/maps.htm

The XML file of all 207 Semeoz’s maps is here: http://www.semeoz.info/download/semeozinfo.maps.xml


(Thomas Kalka) #40

Doing all the screenshots by hand is a big effort, imho, do we really need that ?
I would prefer to find a way to do it automatically.

For mass editing data on the wiki I propose to use the csv export/ import feature.
Basically you can prepare all data, which you would like to go into the wiki in a spreadsheet.

For all data, which you would like to export, one could write a ask query to create an appropriate csv export inside the wiki.


(Maïa Dereva) #47

Not such a big effort, and doing like that, I can check all the maps another time. @alabaeye, I see many sites where there is no map (or site is not working)…

About CSV, I will need help to convert the XML to CSV (as you already done one time). After that, I could prepare the CSV file with the right columns for the import.


(Jon Richter) #50

I have to second @toka here, that if we are going to scrape and generate this for hundreds of pages, we don’t want to rely on a manual process. I will post some ideas for implementation details into https://wekan.indie.host/b/5dgN6K4TkK8ixrDRn/semeoz-transformap/nEnrDMjQe52PoH8fR


(Maïa Dereva) #51

OK, if you prefer, I won’t insist. But I’m not qualified to do that, so I’m going to put me on standby until somebody can help (in particular for the XML to CSV conversion).


(Thomas Kalka) #52

data.csv (134.4 KB)
tags.txt (2.3 KB)
generated by https://github.com/thoka/tm-scrape-semeoz

The next steps could be

@toka should write some helper to check, which entries already exist inside our wiki.

@Maia could find a common keys for these tags (to use them as links into the wiki). This could be either

  • a Wikidata ID (for countries, regions, topics)
  • a link to an appropriate page inside the wiki (you can create pages, if you do not find something appropriate)
  • an english name for the topic

We could start this work in a shared table (google ?) or in base.transformap.co

We also should start another thread on the thumbnail question (and move appropriate parts of this thread therein).


(Thomas Kalka) #55

@Maia
I just imported all pages into the wiki. We will still have to work on tag translation.
See https://wiki.transformaps.net/wiki/Special:RecentChanges


(Maïa Dereva) #56

Wow great!! :smile:
I had too much work last week but I’ll have time to work on this next week.


(Thomas Kalka) #57

@Maia
I just noticed that your export does not include the plain text of the articles, only metadata.
Could you try to produce one including text ?
In addition I ask you to install http://v2.wp-api.org/
This would allow us to extract text (and later push new entries) easily.