Arianne Development Meeting 2016-01-06: Difference between revisions

Content deleted Content added

Inline

Revision as of 13:43, 10 January 2016

We will meet in #arianne on irc.freenode.net, https://webchat.freenode.net/?channels=arianne on Wednesday, 6th January, 2016 at 20.00 server time (19.00 GMT)

Sourceforge, Github, Travis

A quick update from Hendrik about the recent infrastructure changes
...

Prepaing release of Marauroa 4.0

After years of development, perception_json was merged into master.
Release bockers are tagged with MARAUROA_03_99
...

Doing Sendhal releases

At the moment I am the only person who can do a Stendhal release. If I am not available and there is an active contributor, lots of changes pill up. This is hard to review, and I assume it may be frustrating for contributors. Creating a release, however, is both complicated and time consuming.
...

PostgreSQL support

...

Arianne Development Meeting 2016-01-06: Difference between revisions

Revision as of 13:43, 10 January 2016

Contents

Sourceforge, Github, Travis

Prepaing release of Marauroa 4.0

Doing Sendhal releases

PostgreSQL support

Other topics

Navigation menu

@@ Line 5: / Line 5: @@
 * A quick update from Hendrik about the recent infrastructure changes
 * ...
+<!--
+:06hendrik1. Recent infrastructure changes: Sourceforge, Github, Travis
+:07hendrikIn August after an extended downtime of Sourceforge, we created the Arianne organisation on Github and mirrored the Stendhal git repository.
+:07hendrikBack than the mirroring was only from Sourceforge to github, and it was done by the testserver build script once a night.
+:07hendrikSince then we have improved the situation significantly:
+:08hendrikThe mirroring is now done in both directions, and it is done almost instantaniously.
+:08hendrikThis means it does not matter anymore whether commits are done to Sourceforge or Github. So we can use Github features such as pull request. And we can make use of Travis for running the Junit test.
+:09hendrikhttps://stendhalgame.org/wiki/Two_way_git_mirror explains the details.
+:09hendrikIn addition to Stendhal, Marauroa is now on github, too.
+:10hendrikAll junit tests for Marauroa https://travis-ci.org/arianne/marauroa and Stendhal https://travis-ci.org/arianne/stendhal are passing.
+:11alberthjavierhi
+:11hendrikSo before we do a new release, we can just check there. In addition, Travis will execute the test suite for each pull request and post the result directly to the pull request page.
+:12hendrikUnfortunetelly there is nice output of failed tests. So in case of failures, we have to look at the console log or execute the tests locally.
+:12hendrikSo far a quick infrastructure updates. Any comments or questions?
+:12kiherucan it notify about failure to irc like the old jenkins setup?
+:13hendrikAccording to https://docs.travis-ci.com/user/notifications/#IRC-notification that is supposed to work. I tried to configure it, but it does not work.
+:14hendrikhttps://github.com/arianne/stendhal/blob/master/.travis.yml
+:14kiheruaw :-(
+:14markusFirst of all I like travis and the new possibilities of github, esp. pull requests. This brings up a thing we could consider: pointing new contributors to creating pull requests instead of patches?
+:15hendrikI am not sure, if I made a mistake or whether it does not work anymore. In the second case, that's a task for postman.
+:15hendrikmarkus, yes, that may be a good idea.
+:16markusThe question would be: Do we think it is easier for them to create a patch or create a pull request from a fork
+:17markushendrik: didn't you tell me about uploading the testresults somewhere?
+:17hendrikYes, but that only works partly.
+:18markusah, okay. I thought about doing sth like our junit history at work
+:18hendrikBut it would be the place to invoke postman from.
+:18balaurI also like the idea of pull requests more, but for a patch you don't need an account
+:18hendrikYes, that was the idea. But that's quite a bit of work. And it's kind of boring having to develop that again from scratch. So I postponed it.
+:19hendrikPerhaps we should point out both alternatives to new contributors. Or would that be too confusing?
+:19kiheruWe can recommend pull requests, but accept patches as well. For those familiar with github pull requests are easy, but for those who arent, patcvhes are likely simpler
+:19*hendrik nods.
+:19markusyes, kiheru.
+:20balauri agree
+:20hendrikMarkus is refering to our testresults datenbank at work. It basically consists of a table of test runs, a table of test names, and a table of results. The result table links the other two tables.
+:21markusso we should add that to https://stendhalgame.org/wiki/How_To_Create_Patch_For_Stendhal as alternative path.
+:21markusI am thinking about possibities for jenkins to be abused for that.
+:22hendrikSo we can make database queries such as: Which tests failed this time, but were fine on the last run? Or: On which test run did the failure straigh start for this test?
+:22hendrikWe have no jenkins.
+:23markusI know :) But: If we have a "publish-only-reports"-style job, such a jenkins would not waste too much resources
+:24hendrikThis has rather low priority for me, because I assume that we will fix failing tests quickly. Not keep a large number of test unfixed for a long time.
+:24markusJust an idea to get there. But maybe needs more thinking. :)
+:25markusRegarding travis: Can we get the build status prominent in some place?
+:26hendrikhttps://travis-ci.org/arianne/stendhal.svg
+:26markusI was more aiming on embedding that badge in one of our pages
+:27hendrikWhich page?
+:28markusarianne-project.org maybe?
+:28hendrikhttps://stendhalgame.org/wiki/Stendhal_Testing ?
+:28hendrikOn the left site?
+:28markusyes, good idea :)
+:29hendrikAnd a github logo.
+:29markusyes.
+:29kiherucan't hurt to have it there. not as good as screaming about failures at irc, but at least a place where somebody will notice it hopefully
+:30hendrikI think it is "both" not "one or the other".
+:30hendrikOn the website it's basically advertisment that we are doing good work[tm].
+:31hendrikOn IRC is were we will need it for practical reasons.
+:31markusexactly. :)
+:32kiheruagreed. hopefully the irc notification is something that can be fixed
+:33kiheruhey
+:33hendrikIt might be a good idea, if someone else looks at my configuration, to check that I did not mess it up.
+:33kiheruthre travis docs have "irc:" under section "notifications:"
+:34kiheruwhereas on the .travis.yml it is top level, if I read it right
+:35kiheru(I have been trying to find the problem during the discussion. that's the best idea yet)
+:35postmanarianne_rpg: nhnb * rc4c927c49d7c stendhal/.travis.yml: fixed travis irc configuration
+:35hendrikOops.
+:35kiheruI hope it worked
+:35balaurok now let's break a test :p
+:36hendrikShall we move to the next topic?
+:37travis-ciarianne/stendhal#102 (master - c4c927c : Hendrik Brummermann): The build passed.
+:37travis-ciChange view : https://github.com/arianne/stendhal/compare/a0d70ed2e3f1...c4c927c49d7c
+:37travis-ciBuild details : https://travis-ci.org/arianne/stendhal/builds/100660289
+:37kiheruwoo!
+:37markusyay!
+:37hendrikkiheru, yay!!!
+:38postmanarianne_rpg: nhnb * r98affef72de0 stendhal/.classpath: report only test status changes
+:38*hendrik closes Eclipse.
+-->
 == Prepaing release of Marauroa 4.0 ==
@@ Line 11: / Line 88: @@
 * Release bockers are tagged with [https://sourceforge.net/p/arianne/developers/search/?q=labels:%22MARAUROA_03_99%22 MARAUROA_03_99]
 * ...
+<!--
+:39hendrik2. Marauroa 4.0
+:40hendrikA couple of years ago we started to experiment with web clients. We did several attempts using different technologies. To cut a long story short: Only recent browser developments enabled us to build a working webclient.
+:40hendrikAll this time, most of Marauroa's development was done on a branch named perception_json, with only minor changes being backported to master.
+:41travis-ciarianne/stendhal#103 (master - 98affef : Hendrik Brummermann): The build passed.
+:41travis-ciChange view : https://github.com/arianne/stendhal/compare/c4c927c49d7c...98affef72de0
+:41travis-ciBuild details : https://travis-ci.org/arianne/stendhal/builds/100661020
+:41*hendrik glares at travis: That was not suppose to happen. But it has to wait until later.
+:42kiheruwhy not? the config says "on_success: always"
+:42hendrikI changed that with the recent commit to "change2.
+:43kiheruprobably not. you changed stendhal/.classpath, not stendhal/.travis.yml
+:43hendrikargh.
+:44hendrikWithout a real working client, I was afraid that the Marauroa interface were not tested against real requirement.
+:44hendrikBut the last attempt to write a Stendhal webclient is very promissing (although there is still a lot of work required).
+:44hendrikSo I merged the perception_json branch into master, updated CHANGES.txt.
+:45hendrikI think we should do a Marauroa release in the near future.
+:45hendrikhttps://sourceforge.net/p/arianne/developers/search/?q=labels:%22MARAUROA_03_99%22
+:46hendrikThat is a list of tickets concerning aspects that we should think about before releasing.
+:46hendrikMost important is updating documentation and examples.
+:46kiheruprobably should try to make the webclient load the maps from transfercontent so that that part of the json protocol gets tested as well
+:48*hendrik nods: That is one major remaining problem: Stendhal makes use of Marauroas transfer protocol.
+:48hendrikThat is, at least some layers such as tileset and map_data, contain serialized Marauroa objects.
+:49kiheruusing the binary serialization?
+:50hendrikYes :-/
+:51hendrik"typed arrays javascript" is something to look into.
+:51kiheruouch
+:51hendrikI am not sure if that will help.
+:52hendrikWe could blame it on Stendhal. But perhaps there is a way to solve the issue in Marauroa. I am not sure, yet.
+:52kiheruat which point is the serialization format decided? I mean most of the traffic gets serialized to json, but apparently some using the binary
+:53kiheru(I should remember what map_data does, but apparently I don't)
+:53hendrikIt is decided when a Message is sent to the client. The subclasses of Message have both methods.
+:53hendrikSo a json message with the transfer data is sent to the client. This part works.
+:54hendrikBut the data that Stendhal tells Marauroa to transfer, is already binary serialized.
+:55hendrikpublic void transferContent(RPObject target, List<TransferContent> content) {
+:55hendrikin RPServerManager.
+:56hendrikI still need to look at the Stendhal side.
+:58hendrikLet's postpone that until after the meeting.
+:59hendrikAnother open issue is, that I deleted the Maven pom.xml file.
+:59hendrikI was unable to get travis pick up "ant test" with the pom.xml file existing. And "maven test" is not working.
+:00hendrikWhile I don't think that Maven support is a blocker, it would be nice to restore it.
+:00hendrikAnd then there is the breaking change of gameEvents.source (String) to gameEvents.character_id (int).
+:00markusI could look into that. :)
+:01hendrikthank you.
+:01hendrikWhile I prefer to use references, I am not sure about changing the table structure before 4.0
+:02markusIf we want to release soon we should consider delaying that one. Although I think it is a good idea.
+:02hendrikHaving a new major version number is kind of a justification for such changes.
+:02hendrikBut, ... what Markus just said.
+:04hendrikDatabasewise, I would automatically rename the existing gameEvents table and create a new one with the new structure, setting the id to the last id from the old table.
+:05hendrikAs updating the data will likely take a couple of days.
+:05postmanarianne: balaur * developers #274 [Marauroa] Maven for Marauroa http://sf.net/p/arianne/developers/274
+:06hendrikBut there are cases in which source is not refering to player. For example on kill events, source refers to the killer, which may be a creature.
+:09hendrikbalaur raised some other important points about Maven in the linked ticket.
+:09markusyes, saw that :)
+:10hendrikAnything else we need to keep in mind for the Marauroa release?
+:10markusmaking character_id nullable is no option?
+:11hendrikAnd have that in addition to source?
+:13hendrikThis may need some more thinking. I tend to exclude it from 4.0.
+:13hendrikOpinions about including or excluding?
+:14markusI'd go for excluding.
+:14markusMight be an option to make that an addional column.
+:16hendrikOkay, so let's consider that change not a release blocker.
+-->
 == Doing Sendhal releases ==
@@ Line 17: / Line 155: @@
 * At the moment I am the only person who can do a Stendhal release. If I am not available and there is an active contributor, lots of changes pill up. This is hard to review, and I assume it may be frustrating for contributors. Creating a release, however, is both complicated and time consuming.
 * ...
+<!--
+:16hendrik3. Stendhal releases.
+:17hendrikI am concerned about Stendhal releases, and my role in them.
+:18hendrikIn the past, we had releases a lot more often. I mean it is okay not to do a release, if there are no changes.
+:18hendrikBut during the last year, there have been lots of changes by AntumDeluge and soniccuz. And still no timely releases.
+:19hendrikAnd I am the bottleneck here. Doing a release is a lot of work. So if I don't have time, there is no release.
+:20hendrikIn addition to the work of just doing the release, more changes mean more things to test, more time needed. And so a negative feedback look is created.
+:20hendrikloop*
+:23hendrikI like the way AntumDeluge maintained https://stendhalgame.org/wiki/Stendhal_Testing
+:24hendrikSo that it is easy to search, which changes have already been tested.
+:24hendrikto see*
+:24markusyes, that was really a good thing. :)
+:25hendrikPerhaps, I should go into more details, on how a release is created:
+:26hendrikThere are basically 5 phases.
+:26hendrikPhase 0: Preparing the release
+:27hendrik* discuss when to have the next release
+:27hendrik* test all changes and the standard tests for every release
+:27hendrik* remind about CHANGES.txt
+:27hendrik* gather screenshots and think about announcement text.
+:27hendrik* created git branch
+:28hendrikPhase 1: Building and testing of release files
+:29hendrik* this step needs admin permissions
+:29hendrik* checkout the release branch on the build system
+:29hendrik* build the files and updater
+:30hendrik* upload the updater files to the staging area
+:30hendrik* test the updater by using server.update-prop-0.95=http://arianne.sourceforge.net/stendhal/updates/update-0.95.properties.xxx
+:30hendrikPhase 2 and 3: Doing the release for real
+:30hendrik* this is the point of no return
+:31hendrik* upload all files to the correct places
+:31hendrik* update the websites, take the announcement live
+:31hendrik* update the stendhal server and enable the updater
+:31hendrik* restart the server
+:31hendrik* this step needs admin permissions, too.
+:31hendrikPhase 4: After release
+:31hendrik* spread the word
+:32hendrik* update twitter, facebook, sourcefroge news, arianne-announce
+:32hendrik* libregamewiki, gaming.wikia, ...
+:32hendrik-------
+:33hendrikI hope I did not bore you to death.
+:33hendrikSteps 1-3 are a very brief summery, the is a detailed document with all the commands.
+:34hendrikDo you have ideas on how we can get releases more often?
+:37markusAt a first glance: do not put the whole burden on one person. I'll help whenever I can, but I also have not too much time. (might become better in the future)
+:40hendrikPhase 0 and 4 could be put on more shoulders easily, I think.
+:41kiheruI don't have any other ideas. Some of the work is harder to share than others. I can try to help where I can
+:41hendrikthank you.
+:41balaurMaybe it would also help if as many of these steps were automated (create branch, make build, upload, ...). Or are they already?
+:42hendrikOnly a very little. But you are right, it makes sense to put some more effort into automation here.
+:43hendrikSome things are complicated to automate, but it is probably worth to do it anyway.
+:44balaurThe detailed document... is it on the wiki?
+:45hendrikIt is in the admin section, so not really.
+:45hendrikI will sanatize it and move it to the normal wiki.
+:48hendrikI think these steps will help. To summarize: With the document openly available, all developers can help with phase 0. Keeping Stendhal_Testing up date will be helpful too.
+:49hendrikIn the past with two people working together to do a release, putting much effort into automation was not necessary. But I agree that we should do it now.
+:49hendrikIt reduces mistakes, too.
+:51hendrikAnd of course help from kiheru and markus for the admin-permission stuff is much appriciated.
+-->
-== Stendhal Web Client ==
+== PostgreSQL support ==
-Optional topic, depending on how quick we are with the other topics.
 * ...
+<!--
+:51hendrik4. Postgres support
+:51hendrikMarkus?
+:52markusI just had the idea of starting to support postgres.
+:52markusOne thing postgres supports is a JSON datatype for columns which could serve as replacement for the blobs.
+:54hendrikI think adding Postgres support to Marauroa as such is quite simple and little work.
+:54balaurThat would be nice
+:54markusJSON datatype has one advantage to a blob: postgres can issue queries on JSON columns. for example: all json objects with attribute a="blah"
+:54hendrikDoes it support complex nested objects?
+:54markusI agree on that part, hendrik.
+:55hendrikI just realized, that adding postgres support to the stendhal website will be a lot of work. But we have to do that work anyway sooner or later.
+:56hendrikThe website uses deprecated mysql specific php libraries that have been removed in the next php version.
+:56markusthe documentation says, it supports RFC 4627 http://www.postgresql.org/docs/9.3/static/datatype-json.html
+:57markusI think a good first step would be to add plain postgres support before considering JSON
+:57hendrikYes, I agree.
+:58hendrikSo that is: Marauroa, Stendhal, Hall of fame script, website.
+:58hendrikI can do the Marauroa part soon.
+:00markusI could look into Stendhal to see what's needed there.
+:01hendrikI think mixing old php mysql code and new php db code will result in doubling the number of database connectings, and having isolated transactions.
+:02hendrikSo that will be a project for a branch. Luckily we have git nowadays.
+:03hendrikWe can do that in (rather large) chunks by converting all the code that goes to the wiki-database, game-database, website-date.
+-->
 == Other topics ==

Arianne Development Meeting 2016-01-06: Difference between revisions

Revision as of 13:43, 10 January 2016

Sourceforge, Github, Travis

Prepaing release of Marauroa 4.0

Doing Sendhal releases

PostgreSQL support

Other topics

Navigation menu

Search