RNA World (beta)

RNA World (beta) needs donations

The RNA World (beta) project needs donations to cover the BOINC project server (bandwidth and traffic) costs. Please support us and donate!

Posts by Jacob Klein

1) Message boards : News : Manual inspection of the monster WUs: Credits coming up (Message 399)
Posted 1640 days ago by Jacob Klein
You're welcome! I'm looking forward to seeing some results marked as **Validated**! And of course, I'm still trying to help our user base tackle those remaining ~160 VM tasks - I completed 1 today actually. :)

When I first joined RNA World, before you starting using VirtualBox, I couldn't help... because there was no checkpointing, and I restarted my PCs at least monthly but usually even more often.

Now that you use VirtualBox, even though it can be fragile at times, I'm able to help tackle these monsters. I still restart my PCs very often, usually installing new Windows 10 Fast Ring builds every couple of days, but I'm SO GLAD I can participate in this project! I've even learned a ton about VirtualBox, all from your project using it and me trying to keep it working smoothly

So, Thank YOU for the opportunity to learn and help!
2) Message boards : News : Merry Christmas (Message 395)
Posted 1761 days ago by Jacob Klein
Merry Christmas and Happy Holidays, to everyone involved in the RNA World project!
3) Message boards : News : Rechenkraft.net wins Google Impact Challenge 2016 - Thanks to our supporters! (Message 389)
Posted 2063 days ago by Jacob Klein
Congratulations!!
4) Message boards : News : Vbox wrapper updated (Message 384)
Posted 2079 days ago by Jacob Klein
For those tracking this thread...

Christian mentioned that he updated the application to v1.17, which includes an updated VBoxWrapper, and a new vbox.job file that should allow progress to report correctly in the UI, instead of constantly reporting 0.1% progress. Prior versions are still valid for getting work done, though, so DON'T abort your 1.14/1.15/1.16 tasks!

Note: If progress appears to get stuck at 98.765%, it actually is still progressing, DON'T abort it - the estimate was just way too low, but your work is still valid, so don't abort it! You can make sure it is progressing because the progress.txt file's modified date should be getting updated, even if the value inside stays at 0.98765. If you want to know an even crazier way to verify progress, you can PM me and I'll explain a riskier technical way.

Thanks,
Jacob
5) Message boards : News : Vbox wrapper updated (Message 383)
Posted 2085 days ago by Jacob Klein
Hmm... I'm thinking the correct file name is actually "client_state.xml" in your BOINC data directory.

So.. with BOINC closed, find "client_state.xml", and see if you have a "p_vm_extensions_disabled" entry that could be deleted. Take special care when editing this file -- if you get it wrong, you could trash your BOINC projects/tasks.

Let us know what you find!
6) Message boards : News : Vbox wrapper updated (Message 380)
Posted 2088 days ago by Jacob Klein
Is this what you're looking for?
http://github.com/BOINC/boinc/issues/1460
7) Message boards : News : New regular-sized WUs available (Message 352)
Posted 2649 days ago by Jacob Klein
Excellent information. Thank you both!
8) Message boards : News : New regular-sized WUs available (Message 350)
Posted 2650 days ago by Jacob Klein
That's great news, thanks!

Question: Are the new work units VM ones, or not? Server status appears to show several new non-VM S-size short tasks - http://www.rnaworld.de/rnaworld/server_status.php
9) Message boards : News : Latest VirtualBox version now possible (Message 344)
Posted 2901 days ago by Jacob Klein
Crystal Pellet is correct.
4.2.16 will work for both.
4.2.18 and 4.3.0 will not work for RNA World or Test4Theory.
4.3.2 will work for RNA World, but not Test4Theory.

In fact, I believe vboxwrapper will blacklist 4.2.18 and 4.3.0, and send the user a Notice saying that they need to upgrade their VirtualBox.

4.3.2 works fine for RNA World. But if you are also attached to Test4Theory, then it is recommended to keep using 4.2.16 until Oracle VirtualBox Bug 12291 is resolved.

See here for details about the bugs that affect the various VirtualBox versions:
http://lhcathome2.cern.ch/test4theory/forum_thread.php?id=1364&postid=15560#15560

See here for details about the 4.3.2 bug that affects T4T but not RNA World:
https://www.virtualbox.org/ticket/12291
... as of 11/13/2013, Frank says there is a fix checked in, and we'll hopefully see it in the next VirtualBox maintenance release.

Thanks,
Jacob
10) Message boards : News : new Applications using VirtualBox released (Message 339)
Posted 2906 days ago by Jacob Klein
BOINC v7.2.28 has been released to the public today, and is now the recommended version. I'm quite excited about it!

Note: It DID include the VM-task "memory accounting" fix, where BOINC now uses <rsc_memory_bound> when determining how much memory a VirtualBox VM task is consuming. So, it considers 4GB of memory allocated, for each RNA World VM task.

Oh, and the task I was working on? It's still processing, 182 hours [~1 week] done, with an 8-week estimate on a comparable system :) Still crossing my fingers for no errors!

Good luck!
Jacob
11) Message boards : News : new Applications using VirtualBox released (Message 338)
Posted 2921 days ago by Jacob Klein
David and Rom have decided to include the fix, and we are doing additional testing on it.
So far, my testing indicates that 7.2.24+ does indeed use the "4 GB" value when considering memory, and thus won't overcommit beyond the user's specified memory limit.

We haven't publicly released yet, but 7.2.24+ fixes the problem.
And so now, I don't have to use either of the 2 workarounds.

Regards,
Jacob
12) Message boards : News : new Applications using VirtualBox released (Message 337)
Posted 2922 days ago by Jacob Klein
I passed on your message, and will try to reply back here when I know more.

There are actually 2 workarounds for this problem:
1: Use an app_config.xml file and set max_concurrent to a reasonable value. However, this can lead to a work fetch issue that results in an idle CPU.
2: If you only want to run x RNA World VMs, allow work fetch to get x+1 tasks (perhaps by temporarily suspending other projects), and then suspend all but x (and resume all the projects). When an RNA World VM task completes, repeat.

Sure, option 2 is micromanaging a bit, but I'll be using it for the time being, as a workaround that also ensures my CPUs are all kept busy.

Regards,
Jacob
13) Message boards : News : new Applications using VirtualBox released (Message 335)
Posted 2922 days ago by Jacob Klein
Yup, I've implemented the app_config.xml file, thanks.

However, I worked with Rom Walton and David Anderson, and we think we have discovered the cause and fix of my problem. They have supposedly fixed it, but may or may not include it in the upcoming public release.

The issue is that, when calculating how much memory each RNA World VM is using, BOINC used Windows measurements, and came up with ~250 MB, even though the VM "Base Memory" size was 4 GB. The docs (link below) indicate that the 4 GB has to be available as free RAM, and it gets fully-reserved for the client VM, per "Base Memory" documentation in link 2 below. What made matters worse was that Task Manager and Process Explorer both do not show how much host memory a VM has consumed, as evidenced in link 3 below.

I happened to have multiple RNA World VMs running on my 12 GB machine. When I brought in the 4th VM, the Windows UI started becoming erratically non-responsive. Most likely, Windows was paging memory in/out, to give the VMs all 12 GB of my memory. Task manager and Process Explorer were not reporting it as being used, however the UI responsiveness felt terrible at times.

See:
http://superuser.com/questions/66842/how-does-virtualboxs-memory-usage-work
http://www.virtualbox.org/manual/ch03.html#settings-system ("Base Memory" section)
http://forum.sysinternals.com/pe-is-not-showing-all-memory-used-by-virtualbox_topic23886.html

So, David Anderson checked in some code that will, when calculating "running memory" for a VM task, instead of querying Windows on memory usage for processes, will now instead use the <rsc_memory_bound> value as "how much is this VM task using". That way, if I have BOINC set up to use no more than 60% of my 12 GB (which is 7.2 GB), BOINC will not allow a 2nd RNA World VM to launch (since each one now will "constantly consume 4 GB" according to how BOINC calculates running memory for those VM tasks.)

I hope that makes sense. I think we solved that client BOINC bug, but I'm not sure if it will be included in the upcoming public release or not.

Thanks,
Jacob
14) Message boards : News : new Applications using VirtualBox released (Message 333)
Posted 2923 days ago by Jacob Klein
I downloaded a couple more RNA World VMs, and noticed that I get considerable Windows UI contention (where paints/refreshes are lagged/delayed) the more RNA World VMs I run at the same time. I reported the issue to Rom via the BOINC Alpha list.

Anyway, I understand T4T limits to 1 VM per host. I was wondering if you guys might implement an RNA World project setting that, if set by the user, would limit to x per host. I'd prefer to make sure I don't get more than 2 of these non-deterministic VMs at a time.

Regards,
Jacob
15) Message boards : News : new Applications using VirtualBox released (Message 331)
Posted 2924 days ago by Jacob Klein
Okay, thank you.

For your reference, my tasks still count down every second in "Remaining (estimated)", but then that value gets reset every once in a while to be the full value I think.

For instance, I had posted that my VM statuses were:
- [1]: 13,724,158 GFLOPs size, 9.5 hours complete, 17.25 hours remaining, 98.765% progress.
- [2]: 24,969,548 GFLOPs size, 3.5 hours complete, 31.75 hours remaining, 98.765% progress.

Right now (7.5 hours later), my VM statuses are:
- [1]: 13,724,158 GFLOPs size, 17 hours complete, 17.25 hours remaining, 98.765% progress.
- [2]: 24,969,548 GFLOPs size, 10.5 hours complete, 31.75 hours remaining, 98.765% progress.

So, I believe this means that I currently have no means of knowing the real progress at all. :( For all I know, the VMs might be stuck in some sort of infinite loop (are they? What's a realistic completion time for these?)

I hope you are able to significantly improve the indication of progress level, in the next version.

Thanks,
Jacob
16) Message boards : News : new Applications using VirtualBox released (Message 329)
Posted 2924 days ago by Jacob Klein
RNA World:

A huge THANK YOU for implementing checkpointing support via the VirtualBox VM application.
I have now re-enabled RNA World on my main machine, and am presently processing 2 of the VM tasks. So far so good.

Although, it is a bit curious that both of the tasks say "98.765% Progress" constantly with no Progress % updating.
I traced it to 2 possible sources:
- slots\slot#\shared\progress.txt file that has as its contents: 0.98765
- slots\slot#\boinc_task_state.xml file that has: <fraction_done>0.987650</fraction_done>
... either way, it looks like it is hardcoded and not updating.

Also, I see "Remaining (estimated)" decreasing for each second ran, and the total estimated times for my 2 VMs are radically different, presumably due to each one having different "Estimated computation size" values. Here is the status of my 2 VMs:
- [1]: 13,724,158 GFLOPs size, 9.5 hours complete, 17.25 hours remaining, 98.765% progress.
- [2]: 24,969,548 GFLOPs size, 3.5 hours complete, 31.75 hours remaining, 98.765% progress.

So, my questions are:
- How accurate are the "Remaining (estimated)" values for each VM?
- Can anything be done to show actual Progress % in BOINC, with smooth updates to the % value?

Thanks again for your work, I'm happy to be crunching for your project again! --Jacob Klein
17) Message boards : News : Moved XXL workunits (< 72h) to S (Message 312)
Posted 3185 days ago by Jacob Klein
This is bad, because... I had previously changed my settings to not get XXL work units (since I restart my machine often and didn't want to waste all the resources)...

Now you are forcing me to disable the S workunits as well, for the same reason.

A better approach would have been to create a new category for these "medium" workunits (so I could disable mediums while leaving S's enabled)... or to actually implement checkpointing!

I would have preferred a solution that didn't force me to stop doing all work for your project.