cassandra snapshot : Unable to create hard link from errno 1
cassandra, errno
Solution
this seems to be an ubuntu 12.04 security issue on hard-links.
use this command to disable that:
sudo sysctl -w kernel.yama.protected_nonaccess_hardlinks=0
and after restore, reset it to its original value:
sudo sysctl -w kernel.yama.protected_nonaccess_hardlinks=1
Problem
i'm using cassandra 1.2.8 in ubuntu 12.4. i used snapshot command already and backing up process was ok! but now when i use this command i get error like below: ``` Exception in thread "main" FSWriteError in /var/lib/cassandra/data/english/word_doc /snapshots/1393155090911/english-word_doc-ic-10-Index.db at org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:80) at org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1063) at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1566) at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1611) at org.apache.cassandra.db.Table.snapshot(Table.java:194) at org.apache.cassandra.service.StorageService.takeSnapshot(StorageService.java:2205) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:74) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:277) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:253) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:856) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:805) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1476) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:98) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1317) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1409) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:839) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322) at sun.rmi.transport.Transport$1.run(Transport.java:177) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:173) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: java.io.IOException: Unable to create hard link from /var/lib/cassandra/data/english/word_doc/english-word_doc-ic-10-Index.db to /var/lib/cassandra/data/english/word_doc/snapshots/1393155090911/english-word_doc-ic-10-Index.db (errno 1) at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:174) at org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:76) ... 40 more ``` note: my current data base is a back up version that i restore in my cassandra! thanks for your help :) edit: i have two keyspace,A and B. B is a back up version that i restore and store more data on it, but A is not. i can create back up from A with out any problem,but for B i get above error!