CrashPlan Lost My Data and My Business

It's important -- to me at least -- to prepare thorough documentation when a company reaches a certain level of incompetence or creates a certain level of disappointment. I do this for the following reasons:

Very few organizations have disappointed me as much as CrashPlan, with which I was a paying member for over two years. I was never happy (nor even satisfied) with their level of customer service -- but I stayed with them because I thought that their product was sound (which I've now learned was a delusion) and because moving my backup elsewhere would be a challenge (but it was worth it).

The First Problem (2011)

I don't remember the details, and it's dropped from CrashPlan's system -- but I have a strong recollection of being dissatisfied with the response time. At that time, CrashPlan promised paid customers a response in 1 business day, and it took much longer.

The Second Problem (2012 --
Backup Connection Issue; Request #133704

Messages from Me	8
Messages from CrashPlan Telling Me to Wait	2
Messages from CrashPlan Trying to Resolve My Problem	4

I have a CrashPlan+ Unliminted account for my home computer; it is working well.

I also want to back up my work computer to that same home computer (as our department doesn't have a budget for me to establish cloud data storage at this time). I set up the backup-to-a-friend and it consistently works for a few days. Then I start seeing the error message, "Unable to back up - no connection for ____ days".

I suspect this is occurring when my home ISP cycles my dynamic IP address, but I haven't confirmed that suspicion, and I don't know how to get CrashPlan to recover from its connection problem. Can you advise?

I have found information on your site related to connection issues and tried everything it suggests (disabling firewall, establishing telnet connection). This has not resulted in any improvement.

Hello,

I'm David and I'll be looking into your issues. Sorry for the delay and I should have something for you within 24 hours

Thank you again for your patience,
~David

Thanks. I have, by the way, confirmed that the internal and external IP addresses are correct on the source computer (i.e., that they match what CrashPlan reports for the target computer, and also what the computer is reporting through ipconfig). So cycling of the IP address may not be the root of the problem.

Okay, this is getting a little ridiculous. I'm a paid user on my main account, and the new account that I'm trying to back up to my main account's computer was in its trial phase when I submitted this question -- both of which statuses should have entitled me (according to CrashPlan's own documentation and confirmation e-mails) to a response within 1 business day.

I'm a good customer: I pay my bills on time after the first notice. I don't bug you with a lot of questions (I think this is the second issue I've had in 2 years). And I try to do my own research (as I did here) before calling you in for the customer support that my subscription level guarantees me.

But instead of a "1 business day" response, here's the timeline:
(1) I sent my question on Thursday, March 15.
(2) Three business days later, on Tuesday, March 20, I received a reply apologizing for the delay and indicating that I would hear back within 24 hours.
(3) Now it is Thursday, March 29 -- 10 business days after the question and 7 business days (204 hours) -- since I was promised more information "within 24 hours".

I think I've been patient enough, and that now it's time for CrashPlan to make me a priority.

Hello,

I sincerely apologize for the delay. I have extended your subscription by a month due to the delay in responding.

When you see these types of connection issues, they can be a result of a disruption in communications with the server (which would coincide with you getting a new IP address from your ISP). The fastest way to get CrashPlan to reconnect is to do the following:

- Open the CrashPlan application
- Double-click on the CrashPlan logo in the upper right corner
- Enter the following command and hit enter: ra,restart

This will force CrashPlan to reauthorize with the server and restart the CrashPlan backup engine. It will also close the CrashPlan Desktop application. Once it closes the CrashPlan Desktop application, you can reopen it and verify that the connection is re-established.

If this does not resolve the issue, please let me know, as I will then need to take a look at the CrashPlan log files for your computer.

Sincerely,
~David

Unfortunately, it did not work. I'm glad to know about CrashPlan's command window -- but the connection problem still exists.

I didn't know which computer to run it on, or (if both) in what sequence, so I executed the ra,restart command as follows:
(1) On the source (work) computer.
(2) On the destination (home) computer via a remote connection using LogMeIn.
(3) Again on the source (work) computer, in case it should have been run first on the destination (home) computer.

I should mention that we have recently installed a new router (Netgear WNDR3400v2). I think that it was set up before I tried to set up the first (successful) CrashPlan connection -- before CrashPlan lost the connection and was never able to recover it. But I have begun to worry that I am remembering the sequence incorrectly and that this new router may be an important piece of the puzzle. I have looked at the router configuration and I can't find any configuration issues that would seem to explain the problem (i.e., no blocked services or ports) -- but I wanted to mention it in case it is relevant.

And here we are again, one week later this time. What's the deal? I think we're all in agreement that I have paid for a plan that includes next-business-day responses, yet I have received only 2 replies in 21 days -- one of them without any substantive information, but rather only an (inaccurate) promise to follow up within 24 hours. On March 29 I wrote, "I think I've been patient enough, and that now it's time for CrashPlan to make me a priority." That is even more true now on April 5 than it was on March 29.

Please ESCALATE this issue to someone who will find the time to help me with this problem.

Hello Joe,

Thank you for contacting CrashPlan Support.

I have brought your ticket to the attention to a Tier 2 support agent. You will be contacted by this agent shortly.

Thank you,
~Celina

Hello Joe,

My name is Chad, and I'm the agent investigating this issue for you. Can you please send me your log files so I have more information? Here's how:

How It Works - Archive Entire Log Directory
Support needs all the files in the log directory in order to help.
1. In your file system, navigate to the log directory.
• Log file locations
• Windows
• WinXP: C:\Documents and Settings\All Users\Application Data\CrashPlan\log
• Vista: C:\Users\Default\AppData\CrashPlan\log
• Windows7: C:\ProgramData\CrashPlan\Log
• Mac: /Library/Logs/CrashPlan
• From Finder, type: CMD + SHIFT + G and enter /Library/Logs to view the folder.
• Linux (default): /usr/local/crashplan/log
• Solaris (default): /opt/sfw/crashplan/log
2. Archive (create a ZIP file) of the entire the log directory to compress it.
• Mac: Ctrl+click on the log directory and select Compress or Create Archive (10.4)
• Windows: Right-click the log directory, point to Send To and then click Compressed (zipped) Folder

http://support.crashplan.com/doku.php/how_to/send_support_log_files

Please let me know if you have any questions.

Regards,
~Chad

Thank you.

The log files are attached – one for the source computer (containing the files I’m trying to back up) and the other for the destination computer (where the backup will be stored).

Hi Joe,

Thanks for sending me logs! I have bad news--On the Source side, I see errors in the logs, but I *don't* see any errors with CrashPlan itself. There are *tons* of intermittent connectivity errors--lots of timeouts, NAT address translation not working, etc. However, I don't see any actual issues with the client--no java problems, no memory issues, etc. This is good and bad--while everything appears good with the client, the issues seem to be network related, and I can't really offer any solution or fix on it.

On the Destination side, I'm seeing tons more intermittent network errors (very similar to the ones for the source). I'm seeing more of the same, but this might be more specifically related to local network hardware--

[04.18.12 06:36:26.456 INFO W20919435_CDCWorker p42.common.net.ConnectionDiscoveryClient] CDC:: Exception creating NAT-PMP port mapping - java.io.IOException: NapPMP::mapPort(true): No response from router! internalPort=4242, externalPort=7579, lifetime=25200, router
[04.18.12 11:22:08.850 INFO NATContinuation-1882 com.code42.peer.NATContinuation ] [NATContinuation[526829617663075227](1):399922414578303337<->518572328019714971][passive/P_INV_ACK] Unable to reflect - bindLocation=null, rendezvous=RemotePeer-[guid=42, server
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.code42.peer.Reflector.reflect(Reflector.java:159)
at com.code42.peer.PeerConnector.reflect(PeerConnector.java:352)
at com.code42.peer.NATContinuation.reflect(NATContinuation.java:217)
at com.code42.peer.NATContinuation.run(NATContinuation.java:589)
at java.lang.Thread.run(Unknown Source)

I'm also seeing a few errors writing to the local cache on the destination computer--this is generally pretty mild, but it is re-occuring:

[04.18.12 15:41:36.818 WARN MQ-Peer-2 com.code42.io.PropertiesDataFile ] Unable to save properties BackupClientProperties[ C:\Program Files (x86)\CrashPlan\cache\42\cp.properties; {maintenance.runDuration=0, location=C:\Program Files (x86)\CrashPlan\
java.io.FileNotFoundException: C:\Program Files (x86)\CrashPlan\cache\42\cp.properties (The requested operation cannot be performed on a file with a user-mapped section open)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at com.code42.io.PropertiesDataFile.store(PropertiesDataFile.java:147)
at com.code42.backup.manifest.BackupArchiveProperties.store(BackupArchiveProperties.java:515)
at com.code42.backup.BackupClient.setRemoteBackupStats(BackupClient.java:1854)
at com.code42.backup.message.BackupClientMessageReceiver.receiveMessage(BackupClientMessageReceiver.java:169)
at sun.reflect.GeneratedMethodAccessor222.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.code42.messaging.MessageReceiverProxy.receiveMessage(MessageReceiverProxy.java:138)
at com.code42.messaging.Session.receiveMessage(Session.java:283)
at com.code42.messaging.nio.MessageQueue$MessageWorker.handle(MessageQueue.java:145)
at com.code42.messaging.nio.MessageQueue$MessageWorker.handle(MessageQueue.java:81)
at com.code42.queue.QueueWorker.run(QueueWorker.java:60)

Something (perhaps anti-virus/anti-malware) is keeping CrashPlan from being able to write to it's cache. This won't cause the connectivity error you're writing about though.

I'm sorry i can't be more help here--In this situation, I can tell what's wrong, but I can't pinpoint an actual cause. The CrashPlan client and install on both machines appears healthy however. There's definitely something interfering with connectivity between those two devices though, but I can't tell if it's faulty hardware or a software issue that's keeping things from running smoothly.

~Chad

That’s unfortunate. I was hoping that there would be some issue that I could investigate – with guidance from CrashPlan – about my firewall or router configuration. I’ve tried everything I know to facilitate the connection, but I can’t help but suspect that the firewall or router is part of the problem.

I suppose I’ll need to reconsider whether to stay with CrashPlan at all. While my paid subscription is working smoothly, I need to back up the second computer somewhere, and I could apply the annual fee to a multiple-computer account at one of your competitors. I’ll make a decision in the next few days about what to do.

I wish I could tell you that establishing a second paid subscription to CrashPlan was an option under consideration – but as you know I had to wait from March 15 until April 23 (even though I’m promised next-business-day replies as a paid customer) just to get a response of “It’s not going to work, sorry”. This substantially weakens my confidence that CrashPlan will be there for me in the event of a crisis. The only other time I’ve ever had a problem was about 2 years ago, and I recall a similar experience of a frustrating wait that extended into weeks – so I’m skeptical of the claim on your support page that delayed support is the temporary result of a spike in subscriptions.

I could deal with one problem at a time: either (A) slow customer service when things are generally working seamlessly or (B) next-business-day replies to give me some sense of a safety net in the face of a balky connection. But when I can’t get reliability on either side of that equation, my gut tells me I’m setting myself up for an avoidable crisis if I stay with CrashPlan. But I’ll weigh that in more detail over the next few days to avoid making a rash decision.

Hi Joe,

"I could deal with one problem at a time: either (A) slow customer service when things are generally working seamlessly or (B) next-business-day replies to give me some sense of a safety net in the face of a balky connection. But when I can’t get reliability on either side of that equation, my gut tells me I’m setting myself up for an avoidable crisis if I stay with CrashPlan. But I’ll weigh that in more detail over the next few days to avoid making a rash decision."

I can definitely understand your concerns. I am sincerely sorry your ticket took so long to get responded to. As far as network troubleshooting, we do have an article on our web site which has some tips--

http://support.crashplan.com/doku.php/client/troubleshooting/troubleshoot_network_connection

Except that most of these simply will not apply to you, since your connection *is* working--just intermittent. I hope you understand that as a CrashPlan Technical Support agent I cannot troubleshoot your local network connection/hardware for you.

~Chad

The Third and Last Problem (2013)
CrashPlan Lost My Archive (Request #277913)

The Ticket History

Messages from Me	3
Messages from CrashPlan Telling Me to Wait	1
Messages from CrashPlan Trying to Resolve My Problem	2

The CrashPlan application is telling me that it has been unable to back up nor restore my files for 1.9 days. I've checked my firewall and see no problems, and I have made no changes to the computer in that period.

The backup location is CrashPlan Central, and my CrashPlan+ account is current.

What's going on and how do I fix it?

Hi Joe,

Thank you for contacting CrashPlan Support. Thanks also for sharing that screenshot.

In an effort to try to clear up your connection trouble from your CrashPlan desktop application and CrashPlan Central, could you uninstall the CrashPlan desktop application (running the uninstaller from appwiz.cpl). Then swing to http://www.crashplan.com/consumer/download.htmlto get a fresh copy of the installer. After you've got the .exe, please run the installer, and sign back into your account, making sure you select "Adopt" existing backup in the blue/grey window which appears at the top of the app.

If you have any further questions or run in to any trouble, please let me know.

Best,
Shawn

HI Joe,

Just wanted to follow-up after the weekend. I didn't get confirmation you were able to completely uninstall the CrashPlan desktop application, and download/reinstall the latest copy of the CrashPlan application.

How are things coming along?

Best,
Shawn

I had additional information that I thought was best discussed person-to-person.

I called your support line at 6:00 PM Friday evening, and was assured that would call me back before your close-of-business on Friday.

When that didn't happen, I called Saturday morning and was again promised that I would receive a call before your close-of-business.

Two days later, I'm still waiting for that call.

This ticket number was associated to both of those requests, of course.

Hi Joe,

Just checking in to give you an update. I am still looking into your case, and at this time, I have no new information to report.

I will update you again when I gain more information from the engineering team.

Regards,
Shawn

Message I Sent to CrashPlan After Discovering that They Had Closed the Ticket without Resolving It

Wow, so for support issue 277913, here’s the timeline:

March 24, 2013: I noticed that CrashPlan had not been able to connect to my backup archive for several days.

March 26, 2013: I submitted request 277913, asking for guidance on fixing the problem. I received an automated reply that paid customers (of which I am one) could expect replies in 1-2 business days.

March 27, 2013: I received a suggestion from Shawn H. at CrashPlan. This came within the promised period, and I hoped it signaled that my previous issues with CrashPlan response times has been resolved for the better.

March 29, 2013: In the process of trying to implement the suggestion, which included a reinstall of CrashPlan and “adopting” the existing backup, I discovered that there was no existing backup to adopt; it appeared that my entire archive had been lost. I called CrashPlan to understand this situation better, and was told that I would receive a call from Shawn (presumably the same person as the “Shawn H.” who was assigned to the ticket) by the end of the day.

March 30, 2013: Not having heard from Shawn as promised, I called again this next morning, and was again promised I would hear from Shawn by the end of the day. As before, this call never came.

April 1, 2013: Shawn posted an update on the ticket asking about my progress, and I replied saying that I had additional information and questions and that I was still waiting for a phone call to discuss them. Shortly thereafter, I received that phone call from Shawn, who told me that he had never received any of the messages I had left for a call-back. Shawn confirmed that my backup was missing, but said that it was a temporary issue that ordinarily would take a short time, and that he was surprised it had been going on for over a week. He said he would look into it and call me either later that day or the next morning with more information.

April 3, 2013: Shawn never called as promised, but did leave an update to the ticket saying that he was still looking into the issue and would post an update when he got more information from the engineering team.

April 11, 2013: I received an automated e-mail from CrashPlan saying, “We have not heard back from you for about a week on this issue” (despite the fact that I was actually the one who was waiting for a reply). And when I logged in to view the ticket, I found that CrashPlan had closed it. My backup archive is still gone.

Fortunately, I haven’t just been sitting around waiting for CrashPlan; I’ve been using the time to do a little research on my own. This is what I’ve learned:

(1) There have been some publicized instances of CrashPlan’s losing data that sound very similar to my case. One example is detailed at http://jeffreydonenfeld.com/blog/2011/12/crashplan-online-backup-lost-my-entire-backup-archive/. Another is at https://www.facebook.com/CrashPlan/posts/10150413797620124.

(2) CrashPlan will reimburse the prorated balance on the account for dissatisfied users who cancel their CrashPlan accounts.

(3) Carbonite is the same price as CrashPlan for the service level I need.

(4) The CrashPlan application was telling me that it would take over a week to re-establish my backup (and it appeared that this was an accurate estimate based on the slow progress before I interrupted it after 3 days); Carbonite was able to perform a complete backup in just over 48 hours. And in that much faster time, Carbonite was actually backing up more data because it automatically added items that I had not included in my backup set for CrashPlan in addition to all of the items I had explicitly included.

(5) Carbonite’s customer service, while not loved by its users, does allow one to speak with someone who can help, rather than wait for a call-back that might never come.

(6) The previous time I needed customer support from CrashPlan was a year ago (ticket 133704). The events of that interaction largely consisted of increasingly frustrated messages from me that no one was responding to my question at all. Finally, when I got it escalated to someone who would, at least, reply within CrashPlan’s promised response-time window, he was unable to help me with the problem. CrashPlan closed that ticket on April 27, 2012, having communicated with me only 6 times in the 43 days that the ticket was open (and 2 of those communications merely informed me that they were working on it and would be in touch with me later). CrashPlan at this time was promising that paid customers (of which I was one) would receive replies within 1 business day.

I decided to use Carbonite’s 15-day free-trial period as my test for whether to continue putting my trust in CrashPlan. But since CrashPlan has apparently forgotten about me entirely, going so far as to close my ticket without an update, I’m going to declare this experiment over a little early. I’ve never been happy with CrashPlan -- but I stayed with them through inertia, and because I had my historical backup records with them (which, of course, is a moot point now). I hoped and assumed that my previous disappointing experience with CrashPlan support was an anomaly, or that they would recognize their problems and make a change -- but obviously I’m the one who needed to make a change. And now I’m definitely making it.

CrashPlan lost my data.

Then they lost track of my ticket.

And now they’ve lost my business.

I don’t see myself ever coming back to CrashPlan, and I definitely do not -- and will not -- recommend them to others.

CrashPlan

The First Problem (2011)

The Second Problem (2012 -- Backup Connection Issue; Request #133704

The Third and Last Problem (2013) CrashPlan Lost My Archive (Request #277913)

The Ticket History

Message I Sent to CrashPlan After Discovering that They Had Closed the Ticket without Resolving It

The Second Problem (2012 --
Backup Connection Issue; Request #133704

The Third and Last Problem (2013)
CrashPlan Lost My Archive (Request #277913)